__ __ / \ / \ ______ _ _ __ __ __ / /\ / /\ \ / __ \ | | | | / \ \ \ / / / / / / \ \ | | | | | | | | / /\ \ \ / \ \ \ \ / / | |__| | | |__| | / ____ \ | | \ \/ \ \/ / \_ ___/ \____/ /_/ \_\ |_| \__/ \__/ \ \__ \___\ by Red Hat Build, Store, and Distribute your Containers Startup timestamp: Thu May 19 06:08:02 UTC 2022 Running all default registry services without migration Running init script '/quay-registry/conf/init/certs_create.sh' Generating a RSA private key .........................................................++++ .....................++++ writing new private key to 'mitm-key.pem' ----- Running init script '/quay-registry/conf/init/certs_install.sh' Installing extra certificates found in /quay-registry/conf/stack/extra_ca_certs directory Running init script '/quay-registry/conf/init/copy_config_files.sh' Running init script '/quay-registry/conf/init/d_validate_config_bundle.sh' Validating Configuration time="2022-05-19T06:08:03Z" level=debug msg="Validating AccessSettings" time="2022-05-19T06:08:03Z" level=debug msg="Validating ActionLogArchiving" time="2022-05-19T06:08:03Z" level=debug msg="Validating AppTokenAuthentication" time="2022-05-19T06:08:03Z" level=debug msg="Validating BitbucketBuildTrigger" time="2022-05-19T06:08:03Z" level=debug msg="Validating BuildManager" time="2022-05-19T06:08:03Z" level=debug msg="Validating Database" time="2022-05-19T06:08:03Z" level=debug msg="Scheme: postgresql" time="2022-05-19T06:08:03Z" level=debug msg="Host: quayregistry-quay-database:5432" time="2022-05-19T06:08:03Z" level=debug msg="Db: quayregistry-quay-database" time="2022-05-19T06:08:03Z" level=debug msg="Params: " time="2022-05-19T06:08:03Z" level=debug msg="Including params " time="2022-05-19T06:08:03Z" level=debug msg="Pinging database at postgresql://quayregistry-quay-database:fqY8U1REH2bsw1uqxDRbsLeEATE2z80KR1KWB7Y73apHp2W8bXRpYVG4jQYg4jcpBE4HEznmzUXWRgkd@quayregistry-quay-database:5432/quayregistry-quay-database" plpgsql pg_trgm time="2022-05-19T06:08:03Z" level=debug msg="Validating DistributedStorage" time="2022-05-19T06:08:03Z" level=debug msg="Validating ElasticSearch" time="2022-05-19T06:08:03Z" level=debug msg="Validating Email" time="2022-05-19T06:08:03Z" level=debug msg="Validating GitHubBuildTrigger" time="2022-05-19T06:08:03Z" level=debug msg="github response: 200, \n\n\n\n\n\n\n\n \n \n \n \n \n \n \n \n\n\n\n \n \n \n \n\n\n\n\n\n\n \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n \n\n GitHub: Where the world builds software · GitHub\n\n\n\n \n\n \n\n\n \n\n \n \n\n \n\n \n \n \n \n\n\n\n \n\n \n\n\n\n\n \n\n \n \n \n \n \n \n \n \n \n \n \n \n\n\n \n \n\n\n\n \n\n\n \n\n\n \n \n \n \n \n \n\n \n\n \n \n \n\n \n \n \n\n\n\n \n\n\n \n\n \n\n \n\n \n \n \n\n\n\n\n\n \n\n \n\n \n \n\n
\n Skip to content\n \n \n \n \n\n\n \n\n
\n
\n
\n \n \n \n\n \n\n
\n \n\n
\n\n
\n \n Sign up\n \n\n
\n
\n\n
\n
\n
\n\n \n\n \n
\n
\n
\n\n
\n\n
\n\n\n\n\n\n\n\n
\n\n\n \n
\n\n\n \n\n \n\n\n\n\n\n \n
\n\n\n\n\n
\n
\n
\n
\n\n
\n
\n

Where the world builds software

\n

Millions of developers and companies build, ship, and maintain their software on GitHub—the largest and most advanced development platform in the world.

\n\n
\n
\n
\n \n
\n
\n \n
\n
\n \n \n
\n
\n
\n
\n
\n
\n
\n

83+ million

\n

Developers

\n
\n
\n

4+ million

\n

Organizations

\n
\n
\n

200+ million

\n

Repositories

\n
\n
\n

90%

\n

Fortune 100

\n
\n
\n
\n
\n
\n
\n\n
\n
\n
\n \n \n \n\n \n\n
\n
\n\n
\n\n
\n
\n\n \"Glowing\n \n
\n\n
\n
\n
\n
\n \n \n \"Mona\n \n
\n
\n
\n
\n\n \n\n
\n\n
\n
\n
\n
\n
\n

Build like the best with GitHub Enterprise

\n

Take collaboration to the next level with security and administrative features built for teams.

\n \n
\n
\n\n
\n \n \n \n \"\"\n\n
\n
\n
\n
\n\n
\n\n
\n
\n
\n \n\n \n\n \n\n \n
\n
\n\n\n\n
\n \n\n\n
\n
\n
\n
\n
\n

Give your code a home in the cloud

\n
\n
\n\n
\n
    \n
  • \n \n\n

    Record or rewind any change to your code to keep you and your team in sync. Host it all for free with unlimited public and private repositories.

    \n \n Sign up for GitHub \n\n\n
  • \n\n
  • \n \n\n jasonetco added some commits 8 minutes ago\n
  • \n\n
  • \n \n\n \"@jasonetco\"\n Updated README.md\n
  • \n
\n
\n\n
\n
\n
\n
\n \n \n \"The\n \n
\n\n
\n
\n
\n
\n
\n \"File\n
octocat-classifier
\n
\n\n
\n \"File\n
assets
\n
\n\n
\n \"File\n
tests
\n
\n\n
\n \"File\n
LICENSE
\n
\n\n
\n \"File\n
README.md
\n
\n\n
\n \"File\n
index.js
\n
\n\n
\n \"File\n
package.json
\n
\n
\n
\n
\n
\n
\n \"File\n
README.md
\n
\n\n
\n \"File\n
app.js
\n
\n\n
\n \"File\n
index.html
\n
\n
\n
\n
\n
\n          1\n          2\n          3\n          4\n          5\n          6\n          7\n          8\n          9\n          10\n          11\n          12\n          13\n          14\n          15\n          16\n          17\n          18\n          19\n          20\n          21\n          22\n          23\n          24\n          25\n          26\n          27\n          28\n          29\n          30\n          
\n
\n
\n              # Octocat Classifier :octopus: :cat: :mag:\n\n              ![](https://img.shields.io/badge/build-passing-brightgreen) ![](https://img.shields.io/badge/coverage-90%25-green) ![](https://img.shields.io/badge/dependencies-up%20to%20date-brightgreen)\n\n              As the name suggests, Octocat Classifier is used to determine whether a given image contains an Octocat. It is trained with images from the [Octodex](1), images shared with [#MyOctocat on Twitter](2), and [photographs of laptops with :octocat: stickers on them]().\n\n              ## Installation\n\n              ```\n              git clone https://github.com/jasonetco/octocat-classifier\n              ```\n            
\n
\n
\n
\n\n
\n \n
\n\n
\n\n\n
\n \n ~/octocat-classifier\n\n $ gh repo create octocat-classifier\n
\n\n\n
\n\n Created repository jasonetco/octocat-classifier on GitHub\n
\n\n\n \n\n\n
\n\n \u00a0\n
\n\n\n
\n \n ~/octocat-classifier\n\n $ git push origin main\n
\n\n
\n\n
\n
\n\n
\n
\n\n\n \"Light\n
\n
\n
\n\n
\n
    \n
  • \n \n\n \"@jasonetco\"\n Support Octocats shared on Twitter\n
  • \n\n
  • \n \n\n \"@jasonetco\"\n Created index.js\n
  • \n\n
  • \n
    \n
    \n

    Build on what's been built

    \n

    \n Write less code thanks to the world's largest software package registry.\n Find the best community-approved projects to accelerate your work, then share it with the world with npm and GitHub Packages.\n

    \n
    \n\n
    \n
    \n
    \n\n
    \n\n\n
    \n \n ~/octocat-classifier\n\n $ npm install eslint\n
    \n\n\n
    \n\n + eslint@7.8.1\n
    \n\n\n
    \n\n \u00a0\u00a0added 109 packages from 64 contributors and audited 109 packages in 3.491s\n
    \n\n\n
    \n\n \u00a0\n
    \n\n\n
    \n\n 9 packages are looking for funding\n
    \n\n\n
    \n\n \u00a0\u00a0run `npm fund` for details\n
    \n\n\n
    \n\n \u00a0\n
    \n\n\n
    \n\n found 0 vulnerabilities\n
    \n\n\n
    \n \n ~/octocat-classifier\n\n $ \n
    \n\n
    \n\n\n \"Light\n
    \n
    \n
    \n
  • \n\n
  • \n \n\n \"@jasonetco\"\n Added package for javascript linting\n
  • \n
\n
\n
\n
\n
\n\n
\n
\n
\n\n
\n
\n
\n
\n \n\n\n git checkout -b origin add-status-screens\n
\n\n \n\n
\n\n
\n
\n
\n
\n \n\n\n
\n \n \norigin/add-status-screens\n
\n
\n
\n\n
\n
\n \"@nplasterer\"\n \"@ampinsk\"\n \"@joshaber\"\n
\n\n

+3 collaborators

\n
\n\n

Ready player two. Scale your team to any size in the cloud.

\n\n \n Create a new organization for free \n\n\n
\n
\n\n
\n
\n
\n \n \n \"Timeline\n \n
\n
\n
\n\n\n
\n
    \n
  • \n
    \n
    \n \n\n

    Better code starts with pull requests—conversations around your code where you can experiment, squash bugs, and build new features.

    \n
    \n\n
    \n
    \n \n \n \"Pull\n \n
    \n
    \n
    \n
  • \n\n
  • \n
    \n
    \n \n\n

    Code review is built in. Pull requests cover the entire review flow: propose changes, browse code, ask for input, make a suggestion, and sign off in one place.

    \n
    \n
    \n
    \n \n \n \"Pull\n \n
    \n
    \n
    \n
  • \n\n
  • \n
    \n
    \n \n\n

    Know when your pull request is ready to merge when everything’s green. Reviews approved? Check. Tests passing? Check check. No conflicts? Ship it already.

    \n
    \n\n
    \n
    \n \n \n \"Merge\n \n
    \n
    \n
    \n
  • \n\n
  • \n \n\n\n git merge add-status-screens\n
  • \n
\n
\n\n
\n \n\n\n
\n \n\n
\n
\n\n
\n
\n
\n
\n
\n \"iPhone\n\n \n
\n
\n\n
\n

\n Keep work moving.\n Review or merge code, manage notifications, browse repositories, and more with GitHub Mobile.\n

\n

Available for iOS and Android

\n \n
\n
\n\n
\n
\n
\n
\n \n \n \"Viewing\n \n
\n\n
\n
\n
\n
\n \n \n \n
\n\n
\n bash\n
\n
\n\n
\n\n\n
\n \n ~/\n\n $ gh pr status\n
\n\n\n
\n\n Relevant pull requests in cli/cli\n
\n\n\n
\n\n \u00a0\n
\n\n\n
\n\n Current branch\n
\n\n\n
\n\n There is no pull request associated with [main]\n
\n\n\n
\n\n \u00a0\n
\n\n\n
\n\n Created by you\n
\n\n\n
\n\n You have no open pull requests\n
\n\n\n
\n\n \u00a0\n
\n\n\n
\n\n Requesting a code review from you\n
\n\n\n
\n\n #1401 Correctly handle and set empty fields... \n
\n\n\n
\n\n [octocat:emptyBody]\n
\n\n\n
\n\n ✓ Checks passing\n
\n\n\n
\n\n #1357 Added confirmation steps for risk... \n
\n\n\n
\n\n [octocat:confirmations]\n
\n\n\n
\n\n x 1/3 checks failing\n
\n\n\n
\n \n ~/\n\n $ \n
\n\n
\n\n
\n
\n
\n\n
\n

Work however you want.\n Put a GUI on it with GitHub Desktop or stay in the command line with GitHub CLI.\n

\n

Available for macOS, Windows, and Linux*

\n
    \n
  • \"macOS\"
  • \n
  • \"Windows\"
  • \n
  • \"Linux\"
  • \n
\n

* GitHub CLI is available on macOS, Windows, and Linux
* GitHub Desktop is available on macOS and Windows

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n\n
\n\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

Blazing fast cloud developer environments with Codespaces

\n \n Learn more about GitHub Codespaces \n\n\n
\n\n
\n
\n \n\n

The future of code is in the cloud, not your local copy. Codespaces gives you a complete, configurable dev environment on top of a powerful VM in minutes.

\n
\n\n
\n \n\n \n\n

Visual Studio Code, in your browser, full stop. Codespaces brings the world’s most popular desktop editor to every repo. Code, build, test, use the terminal, and open pull requests from anywhere.

\n
\n\n
\n \n\n

Customize to your heart’s desire. Add your favorite VS Code extensions, create a devcontainer config file, install new themes, and tweak your settings.

\n
\n\n
\n
\n
\n \n \n \"VS\n \n \n \n \"VS\n \n \n \n \"VS\n \n
\n\n
\n \"Light\n
\n
\n
\n
\n\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n\n\n
\n
\n
\n
\n\n
\n

Automate anything with GitHub Actions

\n \n Learn more about GitHub Actions \n\n\n
\n\n
\n
\n
\n \n \n
\n Setup CI/CD, enhance your DevOps, and script your entire workflow with GitHub Actions. Kick off automated workflows with GitHub events like push, issue creation, merge, and release.\n
\n\n \n \n \n \n \n
\n
\n\n
\n
\n \n
\n \"GitHub\n \n
\n\n
\n
\n
\n
\n
\n
\n \n \n
\n

11,000+
Actions

\n

Write your own, or import Actions from the open source community, all within our world-class editor. Feeling stuck? Browse the Actions developer docs as you code.

\n \n Explore the Actions Marketplace \n\n\n
\n\n \n \n \n \n \n
\n
\n\n
\n
\n \n\n
\n
\n
\n \n \n \"The\n \n
\n \n \n \"The\n \n
\n\n \n \n \"The\n \n
\n\n
\n
\n
\n
\n
\n

You can have it all. Run actions in any language or operating system, on Linux, macOS, Windows, ARM, and containers. Or all at once with matrix builds.

\n
    \n
  • \"Linux\"
  • \n
  • \"macOS\"
  • \n
  • \"Windows\"
  • \n
  • \"ARM\"
  • \n
\n
\n\n
\n
\n
\n
\n
\n
\n
\n\n \n \n \"Actions\n \n
\n
\n\n
\n
\n
\n \n \n \"Actions\n \n
\n
\n\n
\n
\n\n \n \n \"Actions\n \n
\n
\n\n\n
\n
\n
\n \n \n \"Actions\n \n
\n
\n\n
\n
\n \"\"\n \"\"\n \"\"\n
\n\n
\n \"\"\n \"\"\n \"\"\n
\n\n \n \n \"Actions\n \n
\n
\n
\n
\n \n \n \"Actions\n \n
\n
\n\n
\n
\n
\n \"Large\n
\n \"Large\n
\n\n

With 70 million jobs run per month you’re in good company with Actions, the number one CI service on the world’s largest developer platform.

\n
\n
\n
\n\n
\n
\n
\n

Speaking of automation, Dependabot keeps your projects up to date with automated pull requests that update all your dependencies. Just review and merge to keep your software secure.

\n \n Learn more about Dependabot \n\n\n
\n
\n
\n
\n \n \n \"Automated\n \n
\n \n \n \"Merge\n \n
\n\n
\n
\n
\n\n
\n
\n\n
\n
\n
\n\n
\n
\n\n
\n
\n \n\n
\n\n \n\n
\n\n
\n
\n
\n
\n \n\n
\n
\n\n

Find and fix vulnerabilities
before you merge

\n \n Learn more about advanced security \n\n\n
\n
\n\n
\n
    \n
  • \n
    \n
    \n \n\n

    Secure your code as you write it. CodeQL’s code scanning automatically reviews every change to your codebase and identifies known vulnerabilities before they ever reach production.

    \n
    \n\n
    \n
    \n
    \n \n \n \"Step\n \n
    \n
    \n \n \n \"Step\n \n
    \n
    \n \n \n \"Step\n \n
    \n \n \n \"Code\n \n
    \n
    \n
    \n
  • \n\n
  • \n \n\n Vulnerabilities found\n
  • \n\n
  • \n \n\n \"@jasonetco\"\n Fixed deserialized data security vulnerability\n
  • \n\n
  • \n
    \n
    \n \n\n

    Keep your secrets. We automatically scan repositories for OAuth tokens, API keys, personal tokens, and more. If we find one, we’ll notify you and the partner that issued it to invalidate the secret.

    \n
    \n\n
    \n
    \n \n \n \"An\n \n \n \n \"List\n \n
    \n
    \n
    \n
  • \n\n
  • \n \n\n OAuth token found\n
  • \n\n
  • \n \n\n \"@jasonetco\"\n Replaced OAuth token with a key vault value\n
  • \n
\n
\n\n
\n \n\n\n
\n \n\n
\n
\n\n
\n
\n
\n \n\n

Found a vulnerability? Our security advisory remediation tools help developers identify and disclose them responsibly so maintainers can patch them in dedicated, private workspaces.

\n
\n
\n
\n\n
\n \"Distributed\n
\n
\n
\n
\n
\n
\n\n
\n
\n\n
\n
\n\n
\n
\n

The home for all developers — including you

\n
\n
\n\n
\n
    \n
  • \n \n
    \n \n\n \n GitHub Discussions is dedicated space for your community to come together, ask and answer questions, and have open-ended conversations.\n \n \n Learn how to get started with Discussions \n\n\n
    \n\n
    \n
    \n
    \n \n \n \"Technical\n \n
    \n\n
    \n \"Answered\n \"Floating\n \"Floating\n \"Floating\n\n\n
    \n \n \n \"Answer\n \n
    \n
    \n
    \n
    \n
\n \n\n
  • \n
    \n
    \n \n\n

    \n Amplify your voice in your own personal README on your profile.\n Tell the story of your work through your repositories, contributions, and technologies of choice.\n

    \n
    \n\n
    \n
    \n
    \n \n \n \"GitHub\n \n
    \n\n
    \n \n \n \"GitHub\n \n
    \n
    \n
    \n
    \n
  • \n\n
  • \n \"\"\n \"@sophshep\"\n sophshep started sponsoring you for $10/month 2x 3 days ago\n
  • \n\n
  • \n \"\"\n \"@joshaber\"\n joshaber started sponsoring you for $10/month 5x 2 days ago\n
  • \n\n
  • \n \"\"\n \"@pmarsceill\"\n pmarsceill started sponsoring you for $25/month 2 days ago\n
  • \n\n
  • \n
    \n
    \n \n\n

    That open source code you wrote for yourself might just help someone else. Get paid for building what matters to you with GitHub Sponsors.

    \n\n

    Support the projects you depend on, too.

    \n

    100% to developers, zero fees

    \n \n Learn more about Sponsors \n\n\n
    \n\n
    \n
    \n
    \n \"Floating\n \"Floating\n \"Floating\n\n \n \n \"GitHub\n \n
    \n\n
    \n \"Floating\n \"Floating\n \"Floating\n\n \n \n \"GitHub\n \n
    \n
    \n
    \n
    \n
  • \n \n
    \n\n
    \n
    \n
    \n
    \n\n
    \n \n\n\n
    \n \n \n \n \n \n \n \n\n
    \n
    \n

    Make your contribution

    \n

    Small experiments, inspired inventions, and the software everyone depends on—the code you write on GitHub can reach one codebase or millions.

    \n\n \n Sign up for GitHub\n \n \n\n\n \n Contact Sales\n \n \n\n
    \n\n \n
    \n
    \n
    \n\n
    \n\n
    \n\n \n\n \n\n\n\n\n \n\n
    \n \n \n\n \n \n
    \n \n\n
    \n
    \n
    \n
    \n\n \n\n\n\n\n \n\n\n" time="2022-05-19T06:08:03Z" level=debug msg="Validating GitHubLogin" time="2022-05-19T06:08:03Z" level=debug msg="Validating GitLabBuildTrigger" time="2022-05-19T06:08:03Z" level=debug msg="Validating GoogleLogin" time="2022-05-19T06:08:03Z" level=debug msg="Validating HostSettings" time="2022-05-19T06:08:03Z" level=debug msg="Validating JWTAuthentication" time="2022-05-19T06:08:03Z" level=debug msg="Validating LDAP" time="2022-05-19T06:08:03Z" level=debug msg="Validating OIDC" time="2022-05-19T06:08:03Z" level=debug msg="Validating QuayDocumentation" time="2022-05-19T06:08:03Z" level=debug msg="Validating Redis" time="2022-05-19T06:08:03Z" level=debug msg="Address: quayregistry-quay-redis:6379" time="2022-05-19T06:08:03Z" level=debug msg="Username: " time="2022-05-19T06:08:03Z" level=debug msg="Password Len: 0" time="2022-05-19T06:08:03Z" level=debug msg="Ssl: " time="2022-05-19T06:08:03Z" level=debug msg="Address: quayregistry-quay-redis:6379" time="2022-05-19T06:08:03Z" level=debug msg="Username: " time="2022-05-19T06:08:03Z" level=debug msg="Password Len: 0" time="2022-05-19T06:08:03Z" level=debug msg="Ssl: " time="2022-05-19T06:08:03Z" level=debug msg="Validating RepoMirror" time="2022-05-19T06:08:03Z" level=debug msg="Validating SecurityScanner" time="2022-05-19T06:08:03Z" level=debug msg="Validating TeamSyncing" time="2022-05-19T06:08:03Z" level=debug msg="Validating TimeMachine" time="2022-05-19T06:08:03Z" level=debug msg="Validating UserVisibleSettings" +------------------------+-------+--------+ | Field Group | Error | Status | +------------------------+-------+--------+ | AccessSettings | - | 🟢 | +------------------------+-------+--------+ | ActionLogArchiving | - | 🟢 | +------------------------+-------+--------+ | AppTokenAuthentication | - | 🟢 | +------------------------+-------+--------+ | BitbucketBuildTrigger | - | 🟢 | +------------------------+-------+--------+ | BuildManager | - | 🟢 | +------------------------+-------+--------+ | Database | - | 🟢 | +------------------------+-------+--------+ | DistributedStorage | - | 🟢 | +------------------------+-------+--------+ | ElasticSearch | - | 🟢 | +------------------------+-------+--------+ | Email | - | 🟢 | +------------------------+-------+--------+ | GitHubBuildTrigger | - | 🟢 | +------------------------+-------+--------+ | GitHubLogin | - | 🟢 | +------------------------+-------+--------+ | GitLabBuildTrigger | - | 🟢 | +------------------------+-------+--------+ | GoogleLogin | - | 🟢 | +------------------------+-------+--------+ | HostSettings | - | 🟢 | +------------------------+-------+--------+ | JWTAuthentication | - | 🟢 | +------------------------+-------+--------+ | LDAP | - | 🟢 | +------------------------+-------+--------+ | OIDC | - | 🟢 | +------------------------+-------+--------+ | QuayDocumentation | - | 🟢 | +------------------------+-------+--------+ | Redis | - | 🟢 | +------------------------+-------+--------+ | RepoMirror | - | 🟢 | +------------------------+-------+--------+ | SecurityScanner | - | 🟢 | +------------------------+-------+--------+ | TeamSyncing | - | 🟢 | +------------------------+-------+--------+ | TimeMachine | - | 🟢 | +------------------------+-------+--------+ | UserVisibleSettings | - | 🟢 | +------------------------+-------+--------+ Running init script '/quay-registry/conf/init/nginx_conf_create.sh' Running init script '/quay-registry/conf/init/supervisord_conf_create.sh' Running init script '/quay-registry/conf/init/zz_boot.sh' /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) self._legacy_model = V2SecurityScanner(app, instance_keys, storage) Failed to validate security scanner V2 configuration /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) self._legacy_model = NoopV2SecurityScanner() Generated new service key _IhqQsDfhyqGgRlk0eMHv_QJut6MqBRSuqyYsXKAqds 2022-05-19 06:08:08,023 INFO RPC interface 'supervisor' initialized 2022-05-19 06:08:08,023 CRIT Server 'unix_http_server' running without any HTTP authentication checking 2022-05-19 06:08:08,024 INFO supervisord started with pid 7 2022-05-19 06:08:09,026 INFO spawned: 'stdout' with pid 54 2022-05-19 06:08:09,029 INFO spawned: 'blobuploadcleanupworker' with pid 55 2022-05-19 06:08:09,032 INFO spawned: 'builder' with pid 56 2022-05-19 06:08:09,034 INFO spawned: 'buildlogsarchiver' with pid 57 2022-05-19 06:08:09,037 INFO spawned: 'chunkcleanupworker' with pid 58 2022-05-19 06:08:09,039 INFO spawned: 'dnsmasq' with pid 59 2022-05-19 06:08:09,042 INFO spawned: 'expiredappspecifictokenworker' with pid 60 2022-05-19 06:08:09,044 INFO spawned: 'exportactionlogsworker' with pid 61 2022-05-19 06:08:09,047 INFO spawned: 'gcworker' with pid 62 2022-05-19 06:08:09,050 INFO spawned: 'globalpromstats' with pid 63 2022-05-19 06:08:09,054 INFO spawned: 'gunicorn-registry' with pid 64 2022-05-19 06:08:09,057 INFO spawned: 'gunicorn-secscan' with pid 65 2022-05-19 06:08:09,073 INFO spawned: 'gunicorn-web' with pid 66 2022-05-19 06:08:09,075 INFO spawned: 'jwtproxy' with pid 67 2022-05-19 06:08:09,078 INFO spawned: 'logrotateworker' with pid 68 2022-05-19 06:08:09,080 INFO spawned: 'manifestbackfillworker' with pid 69 2022-05-19 06:08:09,083 INFO spawned: 'memcache' with pid 70 2022-05-19 06:08:09,086 INFO spawned: 'namespacegcworker' with pid 71 2022-05-19 06:08:09,089 INFO spawned: 'nginx' with pid 72 2022-05-19 06:08:09,091 INFO spawned: 'notificationworker' with pid 73 2022-05-19 06:08:09,094 INFO spawned: 'pushgateway' with pid 74 2022-05-19 06:08:09,172 INFO spawned: 'queuecleanupworker' with pid 75 2022-05-19 06:08:09,178 INFO spawned: 'repositoryactioncounter' with pid 79 2022-05-19 06:08:09,184 INFO spawned: 'repositorygcworker' with pid 80 2022-05-19 06:08:09,191 INFO spawned: 'securityscanningnotificationworker' with pid 81 2022-05-19 06:08:09,279 INFO spawned: 'securityworker' with pid 93 2022-05-19 06:08:09,282 INFO spawned: 'servicekey' with pid 94 2022-05-19 06:08:09,378 INFO spawned: 'storagereplication' with pid 97 2022-05-19 06:08:09,391 INFO spawned: 'teamsyncworker' with pid 101 2022-05-19 06:08:10,573 INFO success: stdout entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: blobuploadcleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: builder entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: buildlogsarchiver entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: chunkcleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: dnsmasq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: expiredappspecifictokenworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: exportactionlogsworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: gcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: globalpromstats entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: gunicorn-registry entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: gunicorn-secscan entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: gunicorn-web entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: jwtproxy entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: logrotateworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: manifestbackfillworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: memcache entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: namespacegcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: notificationworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: pushgateway entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: queuecleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: repositoryactioncounter entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: repositorygcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: securityscanningnotificationworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: securityworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: servicekey entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: storagereplication entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-19 06:08:10,574 INFO success: teamsyncworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) dnsmasq stderr | dnsmasq: started, version 2.79 cachesize 150 dnsmasq stderr | dnsmasq: compile time options: IPv6 GNU-getopt DBus no-i18n IDN2 DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth DNSSEC loop-detect inotify dnsmasq stderr | dnsmasq: reading /etc/resolv.conf dnsmasq stderr | dnsmasq: using nameserver 172.30.0.10#53 dnsmasq stderr | dnsmasq: read /etc/hosts - 7 addresses jwtproxy stderr | time="2022-05-19T06:08:09Z" level=info msg="No claims verifiers specified, upstream should be configured to verify authorization" jwtproxy stderr | time="2022-05-19T06:08:09Z" level=info msg="Starting reverse proxy (Listening on 'unix:/tmp/jwtproxy_secscan.sock')" jwtproxy stderr | time="2022-05-19T06:08:09Z" level=info msg="Starting forward proxy (Listening on ':8081')" pushgateway stderr | level=info ts=2022-05-19T06:08:09.384Z caller=main.go:85 msg="starting pushgateway" version="(version=, branch=, revision=)" pushgateway stderr | level=info ts=2022-05-19T06:08:09.385Z caller=main.go:86 build_context="(go=go1.16.12, user=, date=)" pushgateway stderr | level=info ts=2022-05-19T06:08:09.394Z caller=main.go:139 listen_address=:9091 pushgateway stderr | level=info ts=2022-05-19T06:08:09.394Z caller=tls_config.go:191 msg="TLS is disabled." http2=false nginx stdout | 2022/05/19 06:08:09 [alert] 106#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/05/19 06:08:09 [alert] 104#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/05/19 06:08:09 [alert] 111#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/05/19 06:08:09 [alert] 105#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/05/19 06:08:09 [alert] 109#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/05/19 06:08:09 [alert] 110#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/05/19 06:08:09 [alert] 107#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/05/19 06:08:09 [alert] 108#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/05/19 06:08:14 [crit] 106#0: *1 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.129.2.1, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.129.2.84:8080" nginx stdout | 2022/05/19 06:08:14 [crit] 106#0: *1 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.129.2.1, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "10.129.2.84:8080" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:08:14 +0000] "GET /health/instance HTTP/1.1" 502 173 "-" "kube-probe/1.22" (0.000 118 0.000 : 0.000) gunicorn-secscan stdout | 2022-05-19 06:08:17,988 [65] [INFO] [data.registry_model] =============================== gunicorn-secscan stdout | 2022-05-19 06:08:17,988 [65] [INFO] [data.registry_model] Using registry model `` gunicorn-secscan stdout | 2022-05-19 06:08:17,989 [65] [INFO] [data.registry_model] =============================== gunicorn-secscan stdout | 2022-05-19 06:08:19,988 [65] [DEBUG] [app] Loading default config. gunicorn-secscan stdout | 2022-05-19 06:08:19,989 [65] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml gunicorn-secscan stdout | 2022-05-19 06:08:20,088 [65] [DEBUG] [app] Loaded config gunicorn-secscan stdout | 2022-05-19 06:08:20,089 [65] [INFO] [util.ipresolver] Loading AWS IP ranges from disk gunicorn-secscan stdout | 2022-05-19 06:08:20,185 [65] [DEBUG] [util.ipresolver] Building AWS IP ranges exportactionlogsworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) exportactionlogsworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) exportactionlogsworker stderr | Failed to validate security scanner V2 configuration exportactionlogsworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) exportactionlogsworker stderr | self._legacy_model = NoopV2SecurityScanner() gcworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gcworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) gcworker stderr | Failed to validate security scanner V2 configuration gcworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gcworker stderr | self._legacy_model = NoopV2SecurityScanner() gcworker stdout | 2022-05-19 06:08:26,485 [62] [DEBUG] [workers.worker] Scheduling worker. gcworker stdout | 2022-05-19 06:08:26,571 [62] [INFO] [apscheduler.scheduler] Scheduler started gcworker stdout | 2022-05-19 06:08:26,583 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-19 06:08:26,583 [62] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:08:41.584428 gcworker stdout | 2022-05-19 06:08:26,584 [62] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added gcworker stdout | 2022-05-19 06:08:26,992 [62] [INFO] [apscheduler.scheduler] Added job "GarbageCollectionWorker._garbage_collection_repos" to job store "default" gcworker stdout | 2022-05-19 06:08:26,992 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-19 06:08:26,992 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:41.584428+00:00 (in 14.591932 seconds) gunicorn-secscan stdout | 2022-05-19 06:08:27,386 [65] [DEBUG] [util.ipresolver] Finished building AWS IP ranges gunicorn-secscan stdout | 2022-05-19 06:08:28,075 [65] [DEBUG] [data.database] Configuring database gunicorn-secscan stdout | 2022-05-19 06:08:28,080 [65] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-secscan stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-secscan stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) gunicorn-secscan stdout | 2022-05-19 06:08:28,085 [65] [DEBUG] [util.secscan.validator] Missing SECURITY_SCANNER_ENDPOINT configuration gunicorn-secscan stdout | 2022-05-19 06:08:28,085 [65] [WARNING] [data.secscan_model.secscan_v2_model] Failed to validate security scanner V2 configuration gunicorn-secscan stdout | 2022-05-19 06:08:28,085 [65] [INFO] [data.secscan_model] =============================== gunicorn-secscan stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-secscan stderr | self._legacy_model = NoopV2SecurityScanner() gunicorn-secscan stdout | 2022-05-19 06:08:28,085 [65] [INFO] [data.secscan_model] Using split secscan model: `[, ]` gunicorn-secscan stdout | 2022-05-19 06:08:28,090 [65] [INFO] [data.secscan_model] =============================== gunicorn-secscan stdout | 2022-05-19 06:08:28,090 [65] [DEBUG] [data.logs_model] Configuring log lodel gunicorn-secscan stdout | 2022-05-19 06:08:28,090 [65] [INFO] [data.logs_model] =============================== gunicorn-secscan stdout | 2022-05-19 06:08:28,090 [65] [INFO] [data.logs_model] Using logs model `` gunicorn-secscan stdout | 2022-05-19 06:08:28,093 [65] [INFO] [data.logs_model] =============================== gunicorn-secscan stdout | 2022-05-19 06:08:28,284 [65] [DEBUG] [__config__] Starting secscan gunicorn with 2 workers and gevent worker class servicekey stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) servicekey stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) servicekey stderr | Failed to validate security scanner V2 configuration servicekey stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) servicekey stderr | self._legacy_model = NoopV2SecurityScanner() servicekey stdout | 2022-05-19 06:08:28,795 [94] [DEBUG] [workers.worker] Scheduling worker. servicekey stdout | 2022-05-19 06:08:28,796 [94] [INFO] [apscheduler.scheduler] Scheduler started servicekey stdout | 2022-05-19 06:08:28,900 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run servicekey stdout | 2022-05-19 06:08:28,971 [94] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:10:43.972747 servicekey stdout | 2022-05-19 06:08:28,972 [94] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added servicekey stdout | 2022-05-19 06:08:29,094 [94] [INFO] [apscheduler.scheduler] Added job "ServiceKeyWorker._refresh_service_key" to job store "default" servicekey stdout | 2022-05-19 06:08:29,094 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run servicekey stdout | 2022-05-19 06:08:29,094 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:43.972747+00:00 (in 134.877865 seconds) exportactionlogsworker stdout | 2022-05-19 06:08:29,488 [61] [DEBUG] [__main__] Starting export action logs worker exportactionlogsworker stdout | 2022-05-19 06:08:29,776 [61] [DEBUG] [workers.worker] Scheduling worker. exportactionlogsworker stdout | 2022-05-19 06:08:29,776 [61] [INFO] [apscheduler.scheduler] Scheduler started exportactionlogsworker stdout | 2022-05-19 06:08:29,791 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-19 06:08:29,791 [61] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:09:09.792901 exportactionlogsworker stdout | 2022-05-19 06:08:29,795 [61] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added nginx stdout | 2022/05/19 06:08:29 [crit] 106#0: *4 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.129.2.1, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.129.2.84:8080" nginx stdout | 2022/05/19 06:08:29 [crit] 106#0: *4 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.129.2.1, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "10.129.2.84:8080" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:08:29 +0000] "GET /health/instance HTTP/1.1" 502 173 "-" "kube-probe/1.22" (0.000 118 0.000 : 0.000) exportactionlogsworker stdout | 2022-05-19 06:08:30,191 [61] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" exportactionlogsworker stdout | 2022-05-19 06:08:30,191 [61] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:10:58.192705 exportactionlogsworker stdout | 2022-05-19 06:08:30,192 [61] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" exportactionlogsworker stdout | 2022-05-19 06:08:30,192 [61] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:08:36.193076 exportactionlogsworker stdout | 2022-05-19 06:08:30,271 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-19 06:08:30,272 [61] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" exportactionlogsworker stdout | 2022-05-19 06:08:30,273 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:36.193076+00:00 (in 5.920242 seconds) exportactionlogsworker stdout | 2022-05-19 06:08:30,273 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-19 06:08:30,273 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:36.193076+00:00 (in 5.919292 seconds) gunicorn-web stdout | 2022-05-19 06:08:33,888 [66] [INFO] [data.registry_model] =============================== gunicorn-web stdout | 2022-05-19 06:08:33,888 [66] [INFO] [data.registry_model] Using registry model `` gunicorn-web stdout | 2022-05-19 06:08:33,888 [66] [INFO] [data.registry_model] =============================== storagereplication stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) storagereplication stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) storagereplication stderr | Failed to validate security scanner V2 configuration storagereplication stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) storagereplication stderr | self._legacy_model = NoopV2SecurityScanner() storagereplication stdout | 2022-05-19 06:08:35,673 [97] [DEBUG] [__main__] Full storage replication disabled; skipping queuecleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) queuecleanupworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) queuecleanupworker stderr | Failed to validate security scanner V2 configuration queuecleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) queuecleanupworker stderr | self._legacy_model = NoopV2SecurityScanner() exportactionlogsworker stdout | 2022-05-19 06:08:36,193 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-19 06:08:36,286 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:08:36 UTC)" (scheduled at 2022-05-19 06:08:36.193076+00:00) exportactionlogsworker stdout | 2022-05-19 06:08:36,287 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:09.792901+00:00 (in 33.599362 seconds) exportactionlogsworker stdout | 2022-05-19 06:08:36,287 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-19 06:08:36,287 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:09:36 UTC)" executed successfully chunkcleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) chunkcleanupworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) chunkcleanupworker stderr | Failed to validate security scanner V2 configuration chunkcleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) chunkcleanupworker stderr | self._legacy_model = NoopV2SecurityScanner() chunkcleanupworker stdout | 2022-05-19 06:08:36,479 [58] [DEBUG] [__main__] Starting chunk cleanup worker chunkcleanupworker stdout | 2022-05-19 06:08:36,573 [58] [DEBUG] [workers.worker] Scheduling worker. chunkcleanupworker stdout | 2022-05-19 06:08:36,575 [58] [INFO] [apscheduler.scheduler] Scheduler started chunkcleanupworker stdout | 2022-05-19 06:08:36,577 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:08:36,577 [58] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:08:41.578357 chunkcleanupworker stdout | 2022-05-19 06:08:36,578 [58] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added queuecleanupworker stdout | 2022-05-19 06:08:36,673 [75] [DEBUG] [workers.worker] Scheduling worker. queuecleanupworker stdout | 2022-05-19 06:08:36,674 [75] [INFO] [apscheduler.scheduler] Scheduler started queuecleanupworker stdout | 2022-05-19 06:08:36,674 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run queuecleanupworker stdout | 2022-05-19 06:08:36,675 [75] [DEBUG] [workers.worker] First run scheduled for 2022-05-20 04:51:19.676024 queuecleanupworker stdout | 2022-05-19 06:08:36,675 [75] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added chunkcleanupworker stdout | 2022-05-19 06:08:36,776 [58] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" chunkcleanupworker stdout | 2022-05-19 06:08:36,777 [58] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:11:03.778152 chunkcleanupworker stdout | 2022-05-19 06:08:36,777 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:08:36,778 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:41.578357+00:00 (in 4.800357 seconds) chunkcleanupworker stdout | 2022-05-19 06:08:36,778 [58] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" chunkcleanupworker stdout | 2022-05-19 06:08:36,779 [58] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:08:59.780399 chunkcleanupworker stdout | 2022-05-19 06:08:36,779 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:08:36,780 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:41.578357+00:00 (in 4.798125 seconds) chunkcleanupworker stdout | 2022-05-19 06:08:36,781 [58] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" chunkcleanupworker stdout | 2022-05-19 06:08:36,781 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:08:36,781 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:41.578357+00:00 (in 4.796452 seconds) queuecleanupworker stdout | 2022-05-19 06:08:36,786 [75] [INFO] [apscheduler.scheduler] Added job "QueueCleanupWorker._cleanup_queue" to job store "default" queuecleanupworker stdout | 2022-05-19 06:08:36,786 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run queuecleanupworker stdout | 2022-05-19 06:08:36,786 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-20 04:51:19.676024+00:00 (in 81762.889606 seconds) repositoryactioncounter stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) repositoryactioncounter stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) repositoryactioncounter stderr | Failed to validate security scanner V2 configuration repositoryactioncounter stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) repositoryactioncounter stderr | self._legacy_model = NoopV2SecurityScanner() blobuploadcleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) blobuploadcleanupworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) blobuploadcleanupworker stderr | Failed to validate security scanner V2 configuration blobuploadcleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) blobuploadcleanupworker stderr | self._legacy_model = NoopV2SecurityScanner() gunicorn-web stdout | 2022-05-19 06:08:37,780 [66] [DEBUG] [app] Loading default config. gunicorn-web stdout | 2022-05-19 06:08:37,783 [66] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml gunicorn-web stdout | 2022-05-19 06:08:37,896 [66] [DEBUG] [app] Loaded config gunicorn-web stdout | 2022-05-19 06:08:37,974 [66] [INFO] [util.ipresolver] Loading AWS IP ranges from disk gunicorn-web stdout | 2022-05-19 06:08:38,088 [66] [DEBUG] [util.ipresolver] Building AWS IP ranges blobuploadcleanupworker stdout | 2022-05-19 06:08:38,379 [55] [DEBUG] [workers.worker] Scheduling worker. blobuploadcleanupworker stdout | 2022-05-19 06:08:38,380 [55] [INFO] [apscheduler.scheduler] Scheduler started blobuploadcleanupworker stdout | 2022-05-19 06:08:38,393 [55] [DEBUG] [apscheduler.scheduler] Looking for jobs to run blobuploadcleanupworker stdout | 2022-05-19 06:08:38,393 [55] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 07:05:04.394535 blobuploadcleanupworker stdout | 2022-05-19 06:08:38,395 [55] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added teamsyncworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) teamsyncworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) teamsyncworker stderr | Failed to validate security scanner V2 configuration teamsyncworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) teamsyncworker stderr | self._legacy_model = NoopV2SecurityScanner() securityworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) securityworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) securityworker stderr | Failed to validate security scanner V2 configuration securityworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) securityworker stderr | self._legacy_model = NoopV2SecurityScanner() repositoryactioncounter stdout | 2022-05-19 06:08:38,881 [79] [DEBUG] [workers.worker] Scheduling worker. repositoryactioncounter stdout | 2022-05-19 06:08:38,882 [79] [INFO] [apscheduler.scheduler] Scheduler started repositoryactioncounter stdout | 2022-05-19 06:08:38,971 [79] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2022-05-19 06:08:38,972 [79] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 07:50:39.973166 repositoryactioncounter stdout | 2022-05-19 06:08:38,972 [79] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added blobuploadcleanupworker stdout | 2022-05-19 06:08:39,087 [55] [INFO] [apscheduler.scheduler] Added job "BlobUploadCleanupWorker._try_cleanup_uploads" to job store "default" blobuploadcleanupworker stdout | 2022-05-19 06:08:39,087 [55] [DEBUG] [apscheduler.scheduler] Looking for jobs to run blobuploadcleanupworker stdout | 2022-05-19 06:08:39,088 [55] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 07:05:04.394535+00:00 (in 3385.306579 seconds) repositoryactioncounter stdout | 2022-05-19 06:08:39,196 [79] [INFO] [apscheduler.scheduler] Added job "RepositoryActionCountWorker._run_counting" to job store "default" repositoryactioncounter stdout | 2022-05-19 06:08:39,282 [79] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2022-05-19 06:08:39,282 [79] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 07:50:39.973166+00:00 (in 6120.690933 seconds) namespacegcworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) namespacegcworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) namespacegcworker stderr | Failed to validate security scanner V2 configuration namespacegcworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) namespacegcworker stderr | self._legacy_model = NoopV2SecurityScanner() securityworker stdout | 2022-05-19 06:08:39,476 [93] [DEBUG] [workers.worker] Scheduling worker. securityworker stdout | 2022-05-19 06:08:39,477 [93] [INFO] [apscheduler.scheduler] Scheduler started securityworker stdout | 2022-05-19 06:08:39,477 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:08:39,478 [93] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:08:43.478958 securityworker stdout | 2022-05-19 06:08:39,478 [93] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added securityworker stdout | 2022-05-19 06:08:39,687 [93] [INFO] [apscheduler.scheduler] Added job "SecurityWorker._index_in_scanner" to job store "default" securityworker stdout | 2022-05-19 06:08:39,687 [93] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:08:50.688187 securityworker stdout | 2022-05-19 06:08:39,687 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:08:39,687 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:43.478958+00:00 (in 3.791599 seconds) securityworker stdout | 2022-05-19 06:08:39,687 [93] [INFO] [apscheduler.scheduler] Added job "SecurityWorker._index_recent_manifests_in_scanner" to job store "default" securityworker stdout | 2022-05-19 06:08:39,687 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:08:39,687 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:43.478958+00:00 (in 3.791218 seconds) expiredappspecifictokenworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) expiredappspecifictokenworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) expiredappspecifictokenworker stderr | Failed to validate security scanner V2 configuration expiredappspecifictokenworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) expiredappspecifictokenworker stderr | self._legacy_model = NoopV2SecurityScanner() teamsyncworker stdout | 2022-05-19 06:08:39,785 [101] [DEBUG] [__main__] Team syncing is disabled; sleeping expiredappspecifictokenworker stdout | 2022-05-19 06:08:40,277 [60] [DEBUG] [__main__] Starting expired app specific token GC worker expiredappspecifictokenworker stdout | 2022-05-19 06:08:40,294 [60] [DEBUG] [__main__] Found expiration window: 1d expiredappspecifictokenworker stdout | 2022-05-19 06:08:40,372 [60] [DEBUG] [workers.worker] Scheduling worker. expiredappspecifictokenworker stdout | 2022-05-19 06:08:40,373 [60] [INFO] [apscheduler.scheduler] Scheduler started expiredappspecifictokenworker stdout | 2022-05-19 06:08:40,373 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run expiredappspecifictokenworker stdout | 2022-05-19 06:08:40,373 [60] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:19:33.374793 namespacegcworker stdout | 2022-05-19 06:08:40,380 [71] [DEBUG] [__main__] Starting namespace GC worker repositorygcworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) repositorygcworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) repositorygcworker stderr | Failed to validate security scanner V2 configuration repositorygcworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) repositorygcworker stderr | self._legacy_model = NoopV2SecurityScanner() expiredappspecifictokenworker stdout | 2022-05-19 06:08:40,374 [60] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added expiredappspecifictokenworker stdout | 2022-05-19 06:08:40,487 [60] [INFO] [apscheduler.scheduler] Added job "ExpiredAppSpecificTokenWorker._gc_expired_tokens" to job store "default" expiredappspecifictokenworker stdout | 2022-05-19 06:08:40,487 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run expiredappspecifictokenworker stdout | 2022-05-19 06:08:40,487 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:19:33.374793+00:00 (in 652.886856 seconds) namespacegcworker stdout | 2022-05-19 06:08:40,489 [71] [DEBUG] [workers.worker] Scheduling worker. namespacegcworker stdout | 2022-05-19 06:08:40,490 [71] [INFO] [apscheduler.scheduler] Scheduler started namespacegcworker stdout | 2022-05-19 06:08:40,490 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-19 06:08:40,490 [71] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:08:46.491684 namespacegcworker stdout | 2022-05-19 06:08:40,490 [71] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added builder stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) builder stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) builder stderr | Failed to validate security scanner V2 configuration builder stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) builder stderr | self._legacy_model = NoopV2SecurityScanner() namespacegcworker stdout | 2022-05-19 06:08:40,677 [71] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" namespacegcworker stdout | 2022-05-19 06:08:40,677 [71] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:13:21.678921 namespacegcworker stdout | 2022-05-19 06:08:40,678 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-19 06:08:40,678 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:46.491684+00:00 (in 5.813514 seconds) namespacegcworker stdout | 2022-05-19 06:08:40,678 [71] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" namespacegcworker stdout | 2022-05-19 06:08:40,678 [71] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:09:16.679426 namespacegcworker stdout | 2022-05-19 06:08:40,678 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-19 06:08:40,678 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:46.491684+00:00 (in 5.813077 seconds) namespacegcworker stdout | 2022-05-19 06:08:40,678 [71] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" namespacegcworker stdout | 2022-05-19 06:08:40,678 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-19 06:08:40,678 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:46.491684+00:00 (in 5.812796 seconds) repositorygcworker stdout | 2022-05-19 06:08:40,876 [80] [DEBUG] [__main__] Starting repository GC worker repositorygcworker stdout | 2022-05-19 06:08:40,973 [80] [DEBUG] [workers.worker] Scheduling worker. repositorygcworker stdout | 2022-05-19 06:08:40,974 [80] [INFO] [apscheduler.scheduler] Scheduler started repositorygcworker stdout | 2022-05-19 06:08:40,974 [80] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-19 06:08:40,975 [80] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:09:29.976033 builder stdout | 2022-05-19 06:08:40,980 [56] [DEBUG] [__main__] Asking to start build manager with lifecycle "ephemeral" builder stdout | 2022-05-19 06:08:40,980 [56] [DEBUG] [__main__] Will pass buildman hostname quayregistry-quay-builder-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com:443 to builders for websocket connection builder stdout | 2022-05-19 06:08:40,980 [56] [DEBUG] [__main__] Starting build manager with lifecycle "ephemeral" builder stdout | 2022-05-19 06:08:40,980 [56] [DEBUG] [buildman.server] Initializing the lifecycle manager builder stdout | 2022-05-19 06:08:40,980 [56] [DEBUG] [buildman.manager.ephemeral] calling orchestrator_from_config builder stdout | 2022-05-19 06:08:40,981 [56] [DEBUG] [buildman.orchestrator] attempting to create orchestrator redis with kwargs {'host': 'quayregistry-quay-redis', 'password': '', 'skip_keyspace_event_setup': False, 'ssl': False, 'orchestrator_prefix': 'buildman/production/', 'canceller_only': False} builder stdout | 2022-05-19 06:08:40,981 [56] [DEBUG] [buildman.orchestrator] creating pubsub with key buildman/production/orchestrator_events builder stdout | 2022-05-19 06:08:40,987 [56] [DEBUG] [buildman.manager.ephemeral] setting on_key_change callbacks for job expiry, cancel builder stdout | 2022-05-19 06:08:40,987 [56] [DEBUG] [buildman.orchestrator] watching key: buildman/production/building/ builder stdout | 2022-05-19 06:08:40,987 [56] [DEBUG] [buildman.orchestrator] watching key: buildman/production/cancel/ builder stdout | 2022-05-19 06:08:40,987 [56] [DEBUG] [buildman.server] Initializing the gRPC server repositorygcworker stdout | 2022-05-19 06:08:40,975 [80] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added builder stdout | 2022-05-19 06:08:40,992 [56] [DEBUG] [buildman.server] Starting the gRPC server... builder stdout | 2022-05-19 06:08:41,082 [56] [DEBUG] [buildman.server] Starting the build manager... builder stdout | 2022-05-19 06:08:41,082 [56] [DEBUG] [buildman.manager.ephemeral] Initializing work checker builder stdout | 2022-05-19 06:08:41,082 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics repositorygcworker stdout | 2022-05-19 06:08:41,095 [80] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" repositorygcworker stdout | 2022-05-19 06:08:41,095 [80] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:10:19.096808 repositorygcworker stdout | 2022-05-19 06:08:41,171 [80] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-19 06:08:41,172 [80] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" repositorygcworker stdout | 2022-05-19 06:08:41,172 [80] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:09:31.173528 repositorygcworker stdout | 2022-05-19 06:08:41,172 [80] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:29.976033+00:00 (in 48.803624 seconds) repositorygcworker stdout | 2022-05-19 06:08:41,173 [80] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-19 06:08:41,173 [80] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:29.976033+00:00 (in 48.802968 seconds) repositorygcworker stdout | 2022-05-19 06:08:41,173 [80] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" repositorygcworker stdout | 2022-05-19 06:08:41,173 [80] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-19 06:08:41,173 [80] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:29.976033+00:00 (in 48.802271 seconds) builder stdout | 2022-05-19 06:08:41,178 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:08:41,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-19 06:08:41,584 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-registry stdout | 2022-05-19 06:08:41,587 [64] [INFO] [data.registry_model] =============================== gunicorn-registry stdout | 2022-05-19 06:08:41,588 [64] [INFO] [data.registry_model] Using registry model `` gunicorn-registry stdout | 2022-05-19 06:08:41,588 [64] [INFO] [data.registry_model] =============================== chunkcleanupworker stdout | 2022-05-19 06:08:41,588 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:08:41 UTC)" (scheduled at 2022-05-19 06:08:41.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:08:41,588 [58] [DEBUG] [workers.queueworker] Getting work item from queue. globalpromstats stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) globalpromstats stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) globalpromstats stderr | Failed to validate security scanner V2 configuration globalpromstats stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) globalpromstats stderr | self._legacy_model = NoopV2SecurityScanner() gcworker stdout | 2022-05-19 06:08:41,598 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:08:41 UTC)" (scheduled at 2022-05-19 06:08:41.584428+00:00) chunkcleanupworker stdout | 2022-05-19 06:08:41,590 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 8, 41, 589031), True, datetime.datetime(2022, 5, 19, 6, 8, 41, 589031), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:08:41,671 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:51.578357+00:00 (in 9.999499 seconds) gcworker stdout | 2022-05-19 06:08:41,671 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:11.584428+00:00 (in 29.999258 seconds) gcworker stdout | 2022-05-19 06:08:41,674 [62] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) notificationworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) notificationworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) notificationworker stderr | Failed to validate security scanner V2 configuration notificationworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) notificationworker stderr | self._legacy_model = NoopV2SecurityScanner() gcworker stdout | 2022-05-19 06:08:41,692 [62] [DEBUG] [__main__] No GC policies found gcworker stdout | 2022-05-19 06:08:41,692 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-19 06:08:41,692 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:09:11 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:08:41,693 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:08:41,693 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:08:41,693 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:08:51 UTC)" executed successfully logrotateworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) logrotateworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) logrotateworker stderr | Failed to validate security scanner V2 configuration logrotateworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) logrotateworker stderr | self._legacy_model = NoopV2SecurityScanner() globalpromstats stdout | 2022-05-19 06:08:41,891 [63] [DEBUG] [workers.worker] Scheduling worker. globalpromstats stdout | 2022-05-19 06:08:41,893 [63] [INFO] [apscheduler.scheduler] Scheduler started buildlogsarchiver stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) buildlogsarchiver stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) buildlogsarchiver stderr | Failed to validate security scanner V2 configuration buildlogsarchiver stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) buildlogsarchiver stderr | self._legacy_model = NoopV2SecurityScanner() globalpromstats stdout | 2022-05-19 06:08:41,973 [63] [DEBUG] [apscheduler.scheduler] Looking for jobs to run globalpromstats stdout | 2022-05-19 06:08:41,973 [63] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:52:11.974350 securityscanningnotificationworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) securityscanningnotificationworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) securityscanningnotificationworker stderr | Failed to validate security scanner V2 configuration securityscanningnotificationworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) securityscanningnotificationworker stderr | self._legacy_model = NoopV2SecurityScanner() globalpromstats stdout | 2022-05-19 06:08:41,974 [63] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added globalpromstats stdout | 2022-05-19 06:08:42,090 [63] [INFO] [apscheduler.scheduler] Added job "GlobalPrometheusStatsWorker._try_report_stats" to job store "default" globalpromstats stdout | 2022-05-19 06:08:42,091 [63] [DEBUG] [apscheduler.scheduler] Looking for jobs to run globalpromstats stdout | 2022-05-19 06:08:42,092 [63] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:52:11.974350+00:00 (in 2609.882384 seconds) notificationworker stdout | 2022-05-19 06:08:42,295 [73] [DEBUG] [workers.worker] Scheduling worker. notificationworker stdout | 2022-05-19 06:08:42,296 [73] [INFO] [apscheduler.scheduler] Scheduler started notificationworker stdout | 2022-05-19 06:08:42,296 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:08:42,296 [73] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:08:49.297880 logrotateworker stdout | 2022-05-19 06:08:42,298 [68] [DEBUG] [__main__] Action log rotation worker not enabled; skipping notificationworker stdout | 2022-05-19 06:08:42,297 [73] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added securityscanningnotificationworker stdout | 2022-05-19 06:08:42,388 [81] [DEBUG] [__main__] Starting security scanning notification worker notificationworker stdout | 2022-05-19 06:08:42,403 [73] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" notificationworker stdout | 2022-05-19 06:08:42,403 [73] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:09:45.404523 notificationworker stdout | 2022-05-19 06:08:42,403 [73] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" notificationworker stdout | 2022-05-19 06:08:42,404 [73] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:08:48.405016 notificationworker stdout | 2022-05-19 06:08:42,404 [73] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" notificationworker stdout | 2022-05-19 06:08:42,404 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:08:42,404 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:48.405016+00:00 (in 6.000604 seconds) buildlogsarchiver stdout | 2022-05-19 06:08:42,405 [57] [DEBUG] [workers.worker] Scheduling worker. buildlogsarchiver stdout | 2022-05-19 06:08:42,406 [57] [INFO] [apscheduler.scheduler] Scheduler started buildlogsarchiver stdout | 2022-05-19 06:08:42,472 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-19 06:08:42,472 [57] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added buildlogsarchiver stdout | 2022-05-19 06:08:42,472 [57] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:08:45.473591 securityscanningnotificationworker stdout | 2022-05-19 06:08:42,476 [81] [DEBUG] [workers.worker] Scheduling worker. securityscanningnotificationworker stdout | 2022-05-19 06:08:42,477 [81] [INFO] [apscheduler.scheduler] Scheduler started securityscanningnotificationworker stdout | 2022-05-19 06:08:42,478 [81] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-19 06:08:42,478 [81] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added securityscanningnotificationworker stdout | 2022-05-19 06:08:42,478 [81] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:08:51.479280 gunicorn-registry stdout | 2022-05-19 06:08:42,485 [64] [DEBUG] [app] Loading default config. gunicorn-registry stdout | 2022-05-19 06:08:42,485 [64] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml gunicorn-registry stdout | 2022-05-19 06:08:42,503 [64] [DEBUG] [app] Loaded config gunicorn-registry stdout | 2022-05-19 06:08:42,504 [64] [INFO] [util.ipresolver] Loading AWS IP ranges from disk buildlogsarchiver stdout | 2022-05-19 06:08:42,507 [57] [INFO] [apscheduler.scheduler] Added job "ArchiveBuildLogsWorker._archive_redis_buildlogs" to job store "default" buildlogsarchiver stdout | 2022-05-19 06:08:42,507 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-19 06:08:42,507 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:45.473591+00:00 (in 2.965940 seconds) gunicorn-registry stdout | 2022-05-19 06:08:42,579 [64] [DEBUG] [util.ipresolver] Building AWS IP ranges securityscanningnotificationworker stdout | 2022-05-19 06:08:42,582 [81] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" securityscanningnotificationworker stdout | 2022-05-19 06:08:42,582 [81] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:13:42.583285 securityscanningnotificationworker stdout | 2022-05-19 06:08:42,582 [81] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" securityscanningnotificationworker stdout | 2022-05-19 06:08:42,582 [81] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:09:07.583673 securityscanningnotificationworker stdout | 2022-05-19 06:08:42,582 [81] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" securityscanningnotificationworker stdout | 2022-05-19 06:08:42,583 [81] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-19 06:08:42,583 [81] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:51.479280+00:00 (in 8.896023 seconds) manifestbackfillworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) manifestbackfillworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) manifestbackfillworker stderr | Failed to validate security scanner V2 configuration manifestbackfillworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) manifestbackfillworker stderr | self._legacy_model = NoopV2SecurityScanner() gunicorn-web stdout | 2022-05-19 06:08:42,718 [66] [DEBUG] [util.ipresolver] Finished building AWS IP ranges manifestbackfillworker stdout | 2022-05-19 06:08:42,731 [69] [DEBUG] [workers.worker] Scheduling worker. manifestbackfillworker stdout | 2022-05-19 06:08:42,732 [69] [INFO] [apscheduler.scheduler] Scheduler started manifestbackfillworker stdout | 2022-05-19 06:08:42,733 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run manifestbackfillworker stdout | 2022-05-19 06:08:42,733 [69] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added manifestbackfillworker stdout | 2022-05-19 06:08:42,733 [69] [DEBUG] [workers.worker] First run scheduled for 2022-05-19 06:39:07.734495 manifestbackfillworker stdout | 2022-05-19 06:08:42,801 [69] [INFO] [apscheduler.scheduler] Added job "ManifestBackfillWorker._backfill_manifests" to job store "default" manifestbackfillworker stdout | 2022-05-19 06:08:42,801 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run manifestbackfillworker stdout | 2022-05-19 06:08:42,802 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:39:07.734495+00:00 (in 1824.932481 seconds) gunicorn-web stdout | 2022-05-19 06:08:42,823 [66] [DEBUG] [data.database] Configuring database gunicorn-web stdout | 2022-05-19 06:08:42,824 [66] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-web stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) gunicorn-web stdout | 2022-05-19 06:08:42,825 [66] [DEBUG] [util.secscan.validator] Missing SECURITY_SCANNER_ENDPOINT configuration gunicorn-web stdout | 2022-05-19 06:08:42,825 [66] [WARNING] [data.secscan_model.secscan_v2_model] Failed to validate security scanner V2 configuration gunicorn-web stdout | 2022-05-19 06:08:42,826 [66] [INFO] [data.secscan_model] =============================== gunicorn-web stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-web stderr | self._legacy_model = NoopV2SecurityScanner() gunicorn-web stdout | 2022-05-19 06:08:42,826 [66] [INFO] [data.secscan_model] Using split secscan model: `[, ]` gunicorn-web stdout | 2022-05-19 06:08:42,826 [66] [INFO] [data.secscan_model] =============================== gunicorn-web stdout | 2022-05-19 06:08:42,826 [66] [DEBUG] [data.logs_model] Configuring log lodel gunicorn-web stdout | 2022-05-19 06:08:42,826 [66] [INFO] [data.logs_model] =============================== gunicorn-web stdout | 2022-05-19 06:08:42,826 [66] [INFO] [data.logs_model] Using logs model `` gunicorn-web stdout | 2022-05-19 06:08:42,827 [66] [INFO] [data.logs_model] =============================== gunicorn-registry stdout | 2022-05-19 06:08:43,425 [64] [DEBUG] [util.ipresolver] Finished building AWS IP ranges gunicorn-registry stdout | 2022-05-19 06:08:43,477 [64] [DEBUG] [data.database] Configuring database gunicorn-registry stdout | 2022-05-19 06:08:43,479 [64] [INFO] [data.database] Connection pooling enabled for postgresql; stale timeout: None; max connection count: None securityworker stdout | 2022-05-19 06:08:43,479 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-registry stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-registry stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) gunicorn-registry stdout | 2022-05-19 06:08:43,480 [64] [DEBUG] [util.secscan.validator] Missing SECURITY_SCANNER_ENDPOINT configuration gunicorn-registry stdout | 2022-05-19 06:08:43,480 [64] [WARNING] [data.secscan_model.secscan_v2_model] Failed to validate security scanner V2 configuration gunicorn-registry stdout | 2022-05-19 06:08:43,480 [64] [INFO] [data.secscan_model] =============================== gunicorn-registry stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-registry stderr | self._legacy_model = NoopV2SecurityScanner() securityworker stdout | 2022-05-19 06:08:43,480 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:08:43 UTC)" (scheduled at 2022-05-19 06:08:43.478958+00:00) gunicorn-registry stdout | 2022-05-19 06:08:43,480 [64] [INFO] [data.secscan_model] Using split secscan model: `[, ]` gunicorn-registry stdout | 2022-05-19 06:08:43,480 [64] [INFO] [data.secscan_model] =============================== gunicorn-registry stdout | 2022-05-19 06:08:43,480 [64] [DEBUG] [data.logs_model] Configuring log lodel gunicorn-registry stdout | 2022-05-19 06:08:43,480 [64] [INFO] [data.logs_model] =============================== gunicorn-registry stdout | 2022-05-19 06:08:43,480 [64] [INFO] [data.logs_model] Using logs model `` gunicorn-registry stdout | 2022-05-19 06:08:43,481 [64] [INFO] [data.logs_model] =============================== securityworker stdout | 2022-05-19 06:08:43,480 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:08:43,480 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:08:43,481 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:50.688187+00:00 (in 7.208756 seconds) securityworker stdout | 2022-05-19 06:08:43,484 [93] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 securityworker stdout | 2022-05-19 06:08:43,489 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:08:43,492 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:08:43,503 [93] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:08:43,505 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:08:43,505 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:09:13 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:08:43,530 [66] [DEBUG] [__config__] Starting web gunicorn with 4 workers and gevent worker class gunicorn-registry stdout | 2022-05-19 06:08:43,634 [64] [DEBUG] [__config__] Starting registry gunicorn with 8 workers and gevent worker class gunicorn-web stdout | 2022-05-19 06:08:44,917 [231] [DEBUG] [app] Starting request: urn:request:2d959902-65e3-44dc-9e93-0ec06203136a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:08:44,927 [231] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:8080 gunicorn-registry stdout | 2022-05-19 06:08:44,943 [232] [DEBUG] [app] Starting request: urn:request:65d1920c-6004-4df1-a8b1-694a62c863ca (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:08:44,948 [232] [DEBUG] [app] Ending request: urn:request:65d1920c-6004-4df1-a8b1-694a62c863ca (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:65d1920c-6004-4df1-a8b1-694a62c863ca', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:08:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.013 162 0.012) gunicorn-registry stdout | 2022-05-19 06:08:44,950 [232] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:08:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:08:44,951 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:08:44,954 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:08:44,957 [231] [DEBUG] [app] Starting request: urn:request:b374ecc7-c30f-457c-b953-68201047bcf2 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:08:44,959 [231] [DEBUG] [app] Ending request: urn:request:b374ecc7-c30f-457c-b953-68201047bcf2 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b374ecc7-c30f-457c-b953-68201047bcf2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:08:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.004 159 0.004) gunicorn-web stdout | 2022-05-19 06:08:44,960 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:08:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:08:44,961 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:08:44,966 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2022, 5, 19, 6, 8, 44, 964015), None, 'quay', 'quay', datetime.datetime(2022, 5, 19, 6, 8, 44, 964061), 'quay', None, datetime.datetime(2022, 5, 18, 6, 8, 44, 964098), datetime.datetime(2022, 5, 12, 6, 8, 44, 964110), None]) gunicorn-web stdout | 2022-05-19 06:08:44,984 [231] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:08:44,992 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:08:44,995 [231] [DEBUG] [app] Ending request: urn:request:2d959902-65e3-44dc-9e93-0ec06203136a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:2d959902-65e3-44dc-9e93-0ec06203136a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:08:44,996 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:08:44,996 [231] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:08:44 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:08:44 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.085 118 0.086) gunicorn-web stdout | 2022-05-19 06:08:44,999 [231] [DEBUG] [app] Starting request: urn:request:612bb679-6474-438f-9406-ecb20b10955e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-registry stdout | 2022-05-19 06:08:45,007 [241] [DEBUG] [app] Starting request: urn:request:4954267a-dea5-4b9c-8a5b-ac7622e598d4 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:08:45,012 [241] [DEBUG] [app] Ending request: urn:request:4954267a-dea5-4b9c-8a5b-ac7622e598d4 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:4954267a-dea5-4b9c-8a5b-ac7622e598d4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:08:45 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.011 162 0.012) gunicorn-registry stdout | 2022-05-19 06:08:45,014 [241] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:08:45 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:08:45,014 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:08:45,017 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:08:45,027 [233] [DEBUG] [app] Starting request: urn:request:8937f591-5c54-4048-aa37-9679c6db17e3 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:08:45,030 [233] [DEBUG] [app] Ending request: urn:request:8937f591-5c54-4048-aa37-9679c6db17e3 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:8937f591-5c54-4048-aa37-9679c6db17e3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:08:45 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.013 159 0.013) gunicorn-web stdout | 2022-05-19 06:08:45,032 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:08:45,032 [233] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:08:45 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:08:45,034 [231] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:08:45,041 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:08:45,052 [231] [DEBUG] [app] Ending request: urn:request:612bb679-6474-438f-9406-ecb20b10955e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:612bb679-6474-438f-9406-ecb20b10955e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:08:45,052 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:08:45,053 [231] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:08:45 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:08:45 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.055 118 0.055) buildlogsarchiver stdout | 2022-05-19 06:08:45,474 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-19 06:08:45,475 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:08:45 UTC)" (scheduled at 2022-05-19 06:08:45.473591+00:00) buildlogsarchiver stdout | 2022-05-19 06:08:45,477 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 5, 4, 6, 8, 45, 475825), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-19 06:08:45,478 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:15.473591+00:00 (in 29.999412 seconds) buildlogsarchiver stdout | 2022-05-19 06:08:45,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-19 06:08:45,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-19 06:08:45,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:09:15 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-19 06:08:46,200 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} namespacegcworker stdout | 2022-05-19 06:08:46,491 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-19 06:08:46,492 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:08:46 UTC)" (scheduled at 2022-05-19 06:08:46.491684+00:00) namespacegcworker stdout | 2022-05-19 06:08:46,493 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-19 06:08:46,494 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 8, 46, 493096), True, datetime.datetime(2022, 5, 19, 6, 8, 46, 493096), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-19 06:08:46,495 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:16.679426+00:00 (in 30.187283 seconds) namespacegcworker stdout | 2022-05-19 06:08:46,506 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-19 06:08:46,507 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-19 06:08:46,507 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:09:46 UTC)" executed successfully notificationworker stdout | 2022-05-19 06:08:48,405 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:08:48,405 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:08:48 UTC)" (scheduled at 2022-05-19 06:08:48.405016+00:00) notificationworker stdout | 2022-05-19 06:08:48,406 [73] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-19 06:08:48,406 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:08:48 UTC)" executed successfully notificationworker stdout | 2022-05-19 06:08:48,407 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:49.297880+00:00 (in 0.892418 seconds) notificationworker stdout | 2022-05-19 06:08:49,299 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:08:49,300 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:59.297880+00:00 (in 9.997854 seconds) notificationworker stdout | 2022-05-19 06:08:49,300 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:08:59 UTC)" (scheduled at 2022-05-19 06:08:49.297880+00:00) notificationworker stdout | 2022-05-19 06:08:49,300 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:08:49,302 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 8, 49, 300812), True, datetime.datetime(2022, 5, 19, 6, 8, 49, 300812), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:08:49,314 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:08:49,314 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:08:49,314 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:08:59 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-19 06:08:50,112 [188] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '188'} gunicorn-secscan stdout | 2022-05-19 06:08:50,112 [187] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '187'} gunicorn-secscan stdout | 2022-05-19 06:08:50,113 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '65'} securityworker stdout | 2022-05-19 06:08:50,690 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:08:50,690 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:13.478958+00:00 (in 22.788528 seconds) securityworker stdout | 2022-05-19 06:08:50,690 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:09:20 UTC)" (scheduled at 2022-05-19 06:08:50.688187+00:00) securityworker stdout | 2022-05-19 06:08:50,691 [93] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:08:50,691 [93] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2022-05-19 06:08:50,693 [93] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:08:50,693 [93] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2022-05-19 06:08:50,693 [93] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:08:50,693 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:08:50,693 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:08:50,696 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:08:50,698 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:08:50,707 [93] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:08:50,707 [93] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2022-05-19 06:08:50,707 [93] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2022-05-19 06:08:50,708 [93] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2022-05-19 06:08:50,708 [93] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:08:50,710 [93] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:08:50,710 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:08:50,710 [93] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 46, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 251, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2022-05-19 06:08:50,711 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:09:20 UTC)" executed successfully builder stdout | 2022-05-19 06:08:51,189 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:08:51,202 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:08:51,203 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:08:51,213 [56] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-19 06:08:51,479 [81] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-19 06:08:51,480 [81] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:08:51 UTC)" (scheduled at 2022-05-19 06:08:51.479280+00:00) securityscanningnotificationworker stdout | 2022-05-19 06:08:51,480 [81] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-19 06:08:51,482 [81] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 8, 51, 480850), True, datetime.datetime(2022, 5, 19, 6, 8, 51, 480850), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-19 06:08:51,483 [81] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:07.583673+00:00 (in 16.103873 seconds) securityscanningnotificationworker stdout | 2022-05-19 06:08:51,493 [81] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-19 06:08:51,493 [81] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-19 06:08:51,494 [81] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:09:51 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:08:51,672 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:08:51,672 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:08:59.780399+00:00 (in 8.108038 seconds) chunkcleanupworker stdout | 2022-05-19 06:08:51,672 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:01 UTC)" (scheduled at 2022-05-19 06:08:51.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:08:51,673 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:08:51,674 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 8, 51, 673068), True, datetime.datetime(2022, 5, 19, 6, 8, 51, 673068), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:08:51,685 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:08:51,685 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:08:51,686 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:01 UTC)" executed successfully gcworker stdout | 2022-05-19 06:08:52,184 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'gcworker.py', 'pid': '62'} namespacegcworker stdout | 2022-05-19 06:08:53,526 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'namespacegcworker.py', 'pid': '71'} servicekey stdout | 2022-05-19 06:08:53,622 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'servicekeyworker.py', 'pid': '94'} blobuploadcleanupworker stdout | 2022-05-19 06:08:57,820 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} repositoryactioncounter stdout | 2022-05-19 06:08:57,827 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositoryactioncounter.py', 'pid': '79'} storagereplication stdout | 2022-05-19 06:08:58,303 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'storagereplication.py', 'pid': '97'} securityworker stdout | 2022-05-19 06:08:58,690 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityworker.py', 'pid': '93'} notificationworker stdout | 2022-05-19 06:08:59,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:08:59,299 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:09.297880+00:00 (in 9.999079 seconds) notificationworker stdout | 2022-05-19 06:08:59,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:09 UTC)" (scheduled at 2022-05-19 06:08:59.297880+00:00) notificationworker stdout | 2022-05-19 06:08:59,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:08:59,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 8, 59, 299488), True, datetime.datetime(2022, 5, 19, 6, 8, 59, 299488), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:08:59,313 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:08:59,313 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:08:59,313 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:09 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:08:59,781 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:08:59,781 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:01.578357+00:00 (in 1.797027 seconds) chunkcleanupworker stdout | 2022-05-19 06:08:59,781 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:09:59 UTC)" (scheduled at 2022-05-19 06:08:59.780399+00:00) chunkcleanupworker stdout | 2022-05-19 06:08:59,782 [58] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2022-05-19 06:08:59,782 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:09:59 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:08:59,917 [235] [DEBUG] [app] Starting request: urn:request:6af45fa3-8dc3-4e42-b34c-6afb26d759a8 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:08:59,917 [230] [DEBUG] [app] Starting request: urn:request:ad98da48-cda0-48f6-ae8d-628d8bece80d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:08:59,926 [230] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:8080 gunicorn-web stdout | 2022-05-19 06:08:59,926 [235] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:8080 gunicorn-registry stdout | 2022-05-19 06:08:59,935 [241] [DEBUG] [app] Starting request: urn:request:9494c6e8-7c0e-409b-b7da-75cc720d1cdd (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:08:59,936 [241] [DEBUG] [app] Ending request: urn:request:9494c6e8-7c0e-409b-b7da-75cc720d1cdd (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:9494c6e8-7c0e-409b-b7da-75cc720d1cdd', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:08:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.003) gunicorn-registry stdout | 2022-05-19 06:08:59,937 [241] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:08:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:08:59,937 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-19 06:08:59,938 [240] [DEBUG] [app] Starting request: urn:request:39ab8ad4-64c1-46e4-9fe6-b426a6c8b06b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:08:59,941 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:08:59,941 [240] [DEBUG] [app] Ending request: urn:request:39ab8ad4-64c1-46e4-9fe6-b426a6c8b06b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:39ab8ad4-64c1-46e4-9fe6-b426a6c8b06b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:08:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.009 162 0.008) gunicorn-registry stdout | 2022-05-19 06:08:59,942 [240] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:08:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:08:59,943 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:08:59,944 [230] [DEBUG] [app] Starting request: urn:request:2c1eee69-d7ab-460b-bcbe-d8fff537e895 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:08:59,946 [230] [DEBUG] [app] Ending request: urn:request:2c1eee69-d7ab-460b-bcbe-d8fff537e895 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2c1eee69-d7ab-460b-bcbe-d8fff537e895', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:08:59,946 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost nginx stdout | 127.0.0.1 () - - [19/May/2022:06:08:59 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.004 159 0.004) gunicorn-web stdout | 2022-05-19 06:08:59,947 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:08:59 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:08:59,948 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:08:59,948 [235] [DEBUG] [app] Starting request: urn:request:3ef22284-0a49-4700-a35c-cfe39ef02476 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:08:59,950 [235] [DEBUG] [app] Ending request: urn:request:3ef22284-0a49-4700-a35c-cfe39ef02476 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3ef22284-0a49-4700-a35c-cfe39ef02476', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:08:59 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.004 159 0.004) gunicorn-web stdout | 2022-05-19 06:08:59,952 [235] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:08:59 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:08:59,952 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:08:59,952 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2022, 5, 19, 6, 8, 59, 950317), None, 'quay', 'quay', datetime.datetime(2022, 5, 19, 6, 8, 59, 950360), 'quay', None, datetime.datetime(2022, 5, 18, 6, 8, 59, 950400), datetime.datetime(2022, 5, 12, 6, 8, 59, 950416), None]) gunicorn-web stdout | 2022-05-19 06:08:59,957 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2022, 5, 19, 6, 8, 59, 955226), None, 'quay', 'quay', datetime.datetime(2022, 5, 19, 6, 8, 59, 955260), 'quay', None, datetime.datetime(2022, 5, 18, 6, 8, 59, 955291), datetime.datetime(2022, 5, 12, 6, 8, 59, 955303), None]) gunicorn-web stdout | 2022-05-19 06:08:59,968 [230] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:08:59,972 [235] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:08:59,976 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:08:59,979 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:08:59,979 [230] [DEBUG] [app] Ending request: urn:request:ad98da48-cda0-48f6-ae8d-628d8bece80d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ad98da48-cda0-48f6-ae8d-628d8bece80d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:08:59,979 [230] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:08:59,980 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:08:59 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:08:59 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.068 118 0.068) gunicorn-web stdout | 2022-05-19 06:08:59,982 [235] [DEBUG] [app] Ending request: urn:request:6af45fa3-8dc3-4e42-b34c-6afb26d759a8 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6af45fa3-8dc3-4e42-b34c-6afb26d759a8', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:08:59,982 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:08:59,983 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:08:59 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:08:59 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.072 118 0.071) buildlogsarchiver stdout | 2022-05-19 06:09:00,418 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} queuecleanupworker stdout | 2022-05-19 06:09:00,581 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'queuecleanupworker.py', 'pid': '75'} globalpromstats stdout | 2022-05-19 06:09:00,807 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'globalpromstats.py', 'pid': '63'} teamsyncworker stdout | 2022-05-19 06:09:01,109 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'teamsyncworker.py', 'pid': '101'} builder stdout | 2022-05-19 06:09:01,219 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:09:01,234 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:09:01,234 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:09:01,243 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:09:01,579 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:09:01,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:11.578357+00:00 (in 9.999100 seconds) chunkcleanupworker stdout | 2022-05-19 06:09:01,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:11 UTC)" (scheduled at 2022-05-19 06:09:01.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:09:01,580 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:09:01,581 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 9, 1, 580076), True, datetime.datetime(2022, 5, 19, 6, 9, 1, 580076), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:09:01,595 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:09:01,595 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:09:01,595 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:11 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:09:02,614 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} expiredappspecifictokenworker stdout | 2022-05-19 06:09:02,929 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} notificationworker stdout | 2022-05-19 06:09:03,114 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'notificationworker.py', 'pid': '73'} builder stdout | 2022-05-19 06:09:03,119 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'builder.py', 'pid': '56'} repositorygcworker stdout | 2022-05-19 06:09:03,396 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositorygcworker.py', 'pid': '80'} manifestbackfillworker stdout | 2022-05-19 06:09:05,813 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} securityscanningnotificationworker stdout | 2022-05-19 06:09:06,920 [81] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityscanningnotificationworker.py', 'pid': '81'} logrotateworker stdout | 2022-05-19 06:09:07,332 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'logrotateworker.py', 'pid': '68'} securityscanningnotificationworker stdout | 2022-05-19 06:09:07,587 [81] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-19 06:09:07,587 [81] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:51.479280+00:00 (in 43.891788 seconds) securityscanningnotificationworker stdout | 2022-05-19 06:09:07,588 [81] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:07 UTC)" (scheduled at 2022-05-19 06:09:07.583673+00:00) securityscanningnotificationworker stdout | 2022-05-19 06:09:07,588 [81] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-19 06:09:07,588 [81] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:07 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:09:07,987 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '231'} gunicorn-web stdout | 2022-05-19 06:09:07,992 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '230'} gunicorn-web stdout | 2022-05-19 06:09:07,996 [235] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '235'} gunicorn-web stdout | 2022-05-19 06:09:08,002 [233] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '233'} gunicorn-web stdout | 2022-05-19 06:09:08,008 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-19 06:09:09,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:09:09,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:19.297880+00:00 (in 9.999139 seconds) notificationworker stdout | 2022-05-19 06:09:09,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:19 UTC)" (scheduled at 2022-05-19 06:09:09.297880+00:00) notificationworker stdout | 2022-05-19 06:09:09,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:09:09,301 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 9, 9, 299502), True, datetime.datetime(2022, 5, 19, 6, 9, 9, 299502), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:09:09,312 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:09:09,312 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:09:09,313 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:19 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-19 06:09:09,887 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-19 06:09:09,887 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:36.193076+00:00 (in 26.305648 seconds) exportactionlogsworker stdout | 2022-05-19 06:09:09,888 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:09 UTC)" (scheduled at 2022-05-19 06:09:09.792901+00:00) exportactionlogsworker stdout | 2022-05-19 06:09:09,888 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-19 06:09:09,890 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 9, 9, 888432), True, datetime.datetime(2022, 5, 19, 6, 9, 9, 888432), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-19 06:09:09,905 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-19 06:09:09,905 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-19 06:09:09,905 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:09 UTC)" executed successfully builder stdout | 2022-05-19 06:09:11,254 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:09:11,267 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:09:11,267 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:09:11,277 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:09:11,579 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:09:11,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:21.578357+00:00 (in 9.999173 seconds) chunkcleanupworker stdout | 2022-05-19 06:09:11,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:21 UTC)" (scheduled at 2022-05-19 06:09:11.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:09:11,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:09:11,580 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 9, 11, 579850), True, datetime.datetime(2022, 5, 19, 6, 9, 11, 579850), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:09:11,591 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:09:11,591 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:09:11,592 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:21 UTC)" executed successfully gcworker stdout | 2022-05-19 06:09:11,675 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-19 06:09:11,675 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:41.584428+00:00 (in 29.908813 seconds) gcworker stdout | 2022-05-19 06:09:11,676 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:09:41 UTC)" (scheduled at 2022-05-19 06:09:11.584428+00:00) gcworker stdout | 2022-05-19 06:09:11,676 [62] [DEBUG] [__main__] No GC policies found gcworker stdout | 2022-05-19 06:09:11,676 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:09:41 UTC)" executed successfully gunicorn-registry stdout | 2022-05-19 06:09:12,527 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-19 06:09:12,532 [234] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '234'} gunicorn-registry stdout | 2022-05-19 06:09:12,537 [238] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '238'} gunicorn-registry stdout | 2022-05-19 06:09:12,541 [232] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '232'} gunicorn-registry stdout | 2022-05-19 06:09:12,546 [241] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '241'} gunicorn-registry stdout | 2022-05-19 06:09:12,550 [240] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '240'} gunicorn-registry stdout | 2022-05-19 06:09:12,555 [236] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '236'} gunicorn-registry stdout | 2022-05-19 06:09:12,576 [239] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '239'} gunicorn-registry stdout | 2022-05-19 06:09:12,582 [237] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '237'} securityworker stdout | 2022-05-19 06:09:13,479 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:09:13,480 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:20.688187+00:00 (in 7.208339 seconds) securityworker stdout | 2022-05-19 06:09:13,480 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:09:43 UTC)" (scheduled at 2022-05-19 06:09:13.478958+00:00) securityworker stdout | 2022-05-19 06:09:13,480 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:09:13,480 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:09:13,484 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:09:13,486 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:09:13,496 [93] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:09:13,498 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:09:13,499 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:09:43 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:09:14,913 [231] [DEBUG] [app] Starting request: urn:request:7e9f14aa-1d65-40d6-9daf-06544cc064ce (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:09:14,913 [230] [DEBUG] [app] Starting request: urn:request:87c3d83b-b0dc-48fa-bc08-61d1469f1b1a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:09:14,916 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:09:14,916 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:09:14,921 [238] [DEBUG] [app] Starting request: urn:request:56649fbe-4e49-4c38-b4f2-83fe3ce7da13 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:09:14,921 [234] [DEBUG] [app] Starting request: urn:request:4d769ff2-84cd-4a6c-a176-6cc695cbdbd5 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:09:14,923 [238] [DEBUG] [app] Ending request: urn:request:56649fbe-4e49-4c38-b4f2-83fe3ce7da13 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:56649fbe-4e49-4c38-b4f2-83fe3ce7da13', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:09:14,923 [234] [DEBUG] [app] Ending request: urn:request:4d769ff2-84cd-4a6c-a176-6cc695cbdbd5 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:4d769ff2-84cd-4a6c-a176-6cc695cbdbd5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:09:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.007 162 0.007) nginx stdout | 127.0.0.1 () - - [19/May/2022:06:09:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.007 162 0.007) gunicorn-web stdout | 2022-05-19 06:09:14,925 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:09:14,925 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-19 06:09:14,925 [234] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:09:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2022-05-19 06:09:14,926 [238] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:09:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:09:14,927 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:09:14,928 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:09:14,930 [231] [DEBUG] [app] Starting request: urn:request:3ac24f84-7174-478f-8f73-813aeaca9e99 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:09:14,930 [235] [DEBUG] [app] Starting request: urn:request:199bcb7a-0f85-4b17-ab6f-99a8a4102529 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:09:14,931 [231] [DEBUG] [app] Ending request: urn:request:3ac24f84-7174-478f-8f73-813aeaca9e99 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3ac24f84-7174-478f-8f73-813aeaca9e99', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:09:14,931 [235] [DEBUG] [app] Ending request: urn:request:199bcb7a-0f85-4b17-ab6f-99a8a4102529 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:199bcb7a-0f85-4b17-ab6f-99a8a4102529', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:09:14,932 [235] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:09:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:09:14,932 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:09:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:09:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) nginx stdout | 127.0.0.1 () - - [19/May/2022:06:09:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2022-05-19 06:09:14,932 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:09:14,932 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:09:14,933 [231] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:09:14,933 [230] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:09:14,940 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:09:14,940 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:09:14,950 [231] [DEBUG] [app] Ending request: urn:request:7e9f14aa-1d65-40d6-9daf-06544cc064ce (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7e9f14aa-1d65-40d6-9daf-06544cc064ce', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:09:14,950 [230] [DEBUG] [app] Ending request: urn:request:87c3d83b-b0dc-48fa-bc08-61d1469f1b1a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:87c3d83b-b0dc-48fa-bc08-61d1469f1b1a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:09:14,951 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:09:14,951 [230] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:09:14,951 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:09:14 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-19 06:09:14,952 [231] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:09:14 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:09:14 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.040 118 0.040) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:09:14 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.040 118 0.040) buildlogsarchiver stdout | 2022-05-19 06:09:15,478 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-19 06:09:15,479 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:45.473591+00:00 (in 29.994665 seconds) buildlogsarchiver stdout | 2022-05-19 06:09:15,479 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:09:45 UTC)" (scheduled at 2022-05-19 06:09:15.473591+00:00) buildlogsarchiver stdout | 2022-05-19 06:09:15,480 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 5, 4, 6, 9, 15, 479579), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-19 06:09:15,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-19 06:09:15,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-19 06:09:15,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:09:45 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-19 06:09:16,232 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} namespacegcworker stdout | 2022-05-19 06:09:16,682 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-19 06:09:16,683 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:46.491684+00:00 (in 29.808812 seconds) namespacegcworker stdout | 2022-05-19 06:09:16,683 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:16 UTC)" (scheduled at 2022-05-19 06:09:16.679426+00:00) namespacegcworker stdout | 2022-05-19 06:09:16,683 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-19 06:09:16,683 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:16 UTC)" executed successfully notificationworker stdout | 2022-05-19 06:09:19,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:09:19,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:29.297880+00:00 (in 9.999257 seconds) notificationworker stdout | 2022-05-19 06:09:19,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:29 UTC)" (scheduled at 2022-05-19 06:09:19.297880+00:00) notificationworker stdout | 2022-05-19 06:09:19,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:09:19,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 9, 19, 299349), True, datetime.datetime(2022, 5, 19, 6, 9, 19, 299349), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:09:19,320 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:09:19,320 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:09:19,320 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:29 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-19 06:09:20,125 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-19 06:09:20,131 [187] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '187'} gunicorn-secscan stdout | 2022-05-19 06:09:20,137 [188] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '188'} securityworker stdout | 2022-05-19 06:09:20,688 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:09:20,689 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:43.478958+00:00 (in 22.790023 seconds) securityworker stdout | 2022-05-19 06:09:20,689 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:09:50 UTC)" (scheduled at 2022-05-19 06:09:20.688187+00:00) securityworker stdout | 2022-05-19 06:09:20,689 [93] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:09:20,689 [93] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2022-05-19 06:09:20,691 [93] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:09:20,691 [93] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2022-05-19 06:09:20,691 [93] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:09:20,691 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:09:20,691 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:09:20,694 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:09:20,696 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:09:20,705 [93] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:09:20,706 [93] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2022-05-19 06:09:20,706 [93] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2022-05-19 06:09:20,706 [93] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2022-05-19 06:09:20,706 [93] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:09:20,707 [93] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:09:20,707 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:09:20,707 [93] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 46, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 251, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2022-05-19 06:09:20,708 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:09:50 UTC)" executed successfully builder stdout | 2022-05-19 06:09:21,287 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:09:21,300 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:09:21,300 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:09:21,308 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:09:21,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:09:21,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:31.578357+00:00 (in 9.999324 seconds) chunkcleanupworker stdout | 2022-05-19 06:09:21,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:31 UTC)" (scheduled at 2022-05-19 06:09:21.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:09:21,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:09:21,580 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 9, 21, 579759), True, datetime.datetime(2022, 5, 19, 6, 9, 21, 579759), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:09:21,590 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:09:21,591 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:09:21,591 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:31 UTC)" executed successfully gcworker stdout | 2022-05-19 06:09:22,211 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'gcworker.py', 'pid': '62'} namespacegcworker stdout | 2022-05-19 06:09:23,569 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'namespacegcworker.py', 'pid': '71'} servicekey stdout | 2022-05-19 06:09:23,635 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'servicekeyworker.py', 'pid': '94'} blobuploadcleanupworker stdout | 2022-05-19 06:09:27,840 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} repositoryactioncounter stdout | 2022-05-19 06:09:27,846 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositoryactioncounter.py', 'pid': '79'} storagereplication stdout | 2022-05-19 06:09:28,330 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'storagereplication.py', 'pid': '97'} securityworker stdout | 2022-05-19 06:09:28,702 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityworker.py', 'pid': '93'} notificationworker stdout | 2022-05-19 06:09:29,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:09:29,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:39.297880+00:00 (in 9.999221 seconds) notificationworker stdout | 2022-05-19 06:09:29,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:39 UTC)" (scheduled at 2022-05-19 06:09:29.297880+00:00) notificationworker stdout | 2022-05-19 06:09:29,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:09:29,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 9, 29, 299336), True, datetime.datetime(2022, 5, 19, 6, 9, 29, 299336), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:09:29,317 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:09:29,318 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:09:29,318 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:39 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:09:29,913 [235] [DEBUG] [app] Starting request: urn:request:582144ff-b3e1-4e4d-b21d-b34f25763b08 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:09:29,913 [233] [DEBUG] [app] Starting request: urn:request:d54afb9f-a0f5-4639-a3b8-29b976b7335d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:09:29,916 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:09:29,920 [240] [DEBUG] [app] Starting request: urn:request:c6e602ac-b574-45bb-b58a-149e57539d4d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:09:29,921 [240] [DEBUG] [app] Ending request: urn:request:c6e602ac-b574-45bb-b58a-149e57539d4d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c6e602ac-b574-45bb-b58a-149e57539d4d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:09:29,922 [240] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:09:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:09:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.004 162 0.003) gunicorn-web stdout | 2022-05-19 06:09:29,922 [233] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:8080 gunicorn-web stdout | 2022-05-19 06:09:29,922 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-19 06:09:29,925 [240] [DEBUG] [app] Starting request: urn:request:f3f523c7-ca80-40c8-9700-26258367081d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:09:29,925 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:09:29,926 [240] [DEBUG] [app] Ending request: urn:request:f3f523c7-ca80-40c8-9700-26258367081d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f3f523c7-ca80-40c8-9700-26258367081d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:09:29,927 [240] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:09:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:09:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.003) gunicorn-web stdout | 2022-05-19 06:09:29,927 [233] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:09:29,928 [231] [DEBUG] [app] Starting request: urn:request:6cc6d5a2-eed7-45aa-98c1-481c2bcaa078 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:09:29,929 [231] [DEBUG] [app] Ending request: urn:request:6cc6d5a2-eed7-45aa-98c1-481c2bcaa078 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6cc6d5a2-eed7-45aa-98c1-481c2bcaa078', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:09:29,929 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:09:29 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:09:29 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2022-05-19 06:09:29,930 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:09:29,930 [233] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:09:29,930 [235] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:09:29,932 [231] [DEBUG] [app] Starting request: urn:request:b41d5c42-09b7-4d6c-a7ea-c8ef1665e6b4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:09:29,933 [231] [DEBUG] [app] Ending request: urn:request:b41d5c42-09b7-4d6c-a7ea-c8ef1665e6b4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b41d5c42-09b7-4d6c-a7ea-c8ef1665e6b4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:09:29,933 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:09:29 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:09:29 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2022-05-19 06:09:29,933 [233] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:09:29,937 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2022, 5, 19, 6, 9, 29, 935751), None, 'quay', 'quay', datetime.datetime(2022, 5, 19, 6, 9, 29, 935837), 'quay', None, datetime.datetime(2022, 5, 18, 6, 9, 29, 935859), datetime.datetime(2022, 5, 12, 6, 9, 29, 935871), None]) gunicorn-web stdout | 2022-05-19 06:09:29,938 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:09:29,948 [235] [DEBUG] [app] Ending request: urn:request:582144ff-b3e1-4e4d-b21d-b34f25763b08 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:582144ff-b3e1-4e4d-b21d-b34f25763b08', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:09:29,949 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:09:29,949 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:09:29 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:09:29 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.038 118 0.038) gunicorn-web stdout | 2022-05-19 06:09:29,951 [233] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:09:29,959 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:09:29,962 [233] [DEBUG] [app] Ending request: urn:request:d54afb9f-a0f5-4639-a3b8-29b976b7335d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d54afb9f-a0f5-4639-a3b8-29b976b7335d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:09:29,963 [233] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:09:29,963 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:09:29 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:09:29 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.052 118 0.052) repositorygcworker stdout | 2022-05-19 06:09:29,976 [80] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-19 06:09:29,977 [80] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:09:29 UTC)" (scheduled at 2022-05-19 06:09:29.976033+00:00) repositorygcworker stdout | 2022-05-19 06:09:29,978 [80] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-19 06:09:29,980 [80] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 9, 29, 978156), True, datetime.datetime(2022, 5, 19, 6, 9, 29, 978156), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-19 06:09:29,981 [80] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:31.173528+00:00 (in 1.196988 seconds) repositorygcworker stdout | 2022-05-19 06:09:29,993 [80] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-19 06:09:29,993 [80] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-19 06:09:29,993 [80] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:29 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-19 06:09:30,449 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} queuecleanupworker stdout | 2022-05-19 06:09:30,621 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'queuecleanupworker.py', 'pid': '75'} globalpromstats stdout | 2022-05-19 06:09:30,847 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'globalpromstats.py', 'pid': '63'} teamsyncworker stdout | 2022-05-19 06:09:31,145 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'teamsyncworker.py', 'pid': '101'} repositorygcworker stdout | 2022-05-19 06:09:31,178 [80] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-19 06:09:31,179 [80] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:19.096808+00:00 (in 47.917802 seconds) repositorygcworker stdout | 2022-05-19 06:09:31,179 [80] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:31 UTC)" (scheduled at 2022-05-19 06:09:31.173528+00:00) repositorygcworker stdout | 2022-05-19 06:09:31,179 [80] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-19 06:09:31,179 [80] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:31 UTC)" executed successfully builder stdout | 2022-05-19 06:09:31,318 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:09:31,333 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:09:31,333 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:09:31,344 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:09:31,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:09:31,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:41.578357+00:00 (in 9.999117 seconds) chunkcleanupworker stdout | 2022-05-19 06:09:31,580 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:41 UTC)" (scheduled at 2022-05-19 06:09:31.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:09:31,580 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:09:31,582 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 9, 31, 580446), True, datetime.datetime(2022, 5, 19, 6, 9, 31, 580446), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:09:31,595 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:09:31,595 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:09:31,595 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:41 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:09:32,641 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} expiredappspecifictokenworker stdout | 2022-05-19 06:09:32,946 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} notificationworker stdout | 2022-05-19 06:09:33,133 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'notificationworker.py', 'pid': '73'} builder stdout | 2022-05-19 06:09:33,155 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'builder.py', 'pid': '56'} repositorygcworker stdout | 2022-05-19 06:09:33,420 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositorygcworker.py', 'pid': '80'} manifestbackfillworker stdout | 2022-05-19 06:09:35,837 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} exportactionlogsworker stdout | 2022-05-19 06:09:36,193 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-19 06:09:36,194 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:09.792901+00:00 (in 33.598774 seconds) exportactionlogsworker stdout | 2022-05-19 06:09:36,194 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:36 UTC)" (scheduled at 2022-05-19 06:09:36.193076+00:00) exportactionlogsworker stdout | 2022-05-19 06:09:36,194 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-19 06:09:36,194 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:36 UTC)" executed successfully securityscanningnotificationworker stdout | 2022-05-19 06:09:36,962 [81] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityscanningnotificationworker.py', 'pid': '81'} logrotateworker stdout | 2022-05-19 06:09:37,369 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'logrotateworker.py', 'pid': '68'} gunicorn-web stdout | 2022-05-19 06:09:38,001 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '231'} gunicorn-web stdout | 2022-05-19 06:09:38,008 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '230'} gunicorn-web stdout | 2022-05-19 06:09:38,016 [235] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '235'} gunicorn-web stdout | 2022-05-19 06:09:38,021 [233] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '233'} gunicorn-web stdout | 2022-05-19 06:09:38,029 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-19 06:09:39,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:09:39,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:45.404523+00:00 (in 6.105908 seconds) notificationworker stdout | 2022-05-19 06:09:39,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:49 UTC)" (scheduled at 2022-05-19 06:09:39.297880+00:00) notificationworker stdout | 2022-05-19 06:09:39,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:09:39,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 9, 39, 299292), True, datetime.datetime(2022, 5, 19, 6, 9, 39, 299292), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:09:39,318 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:09:39,318 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:09:39,318 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:49 UTC)" executed successfully builder stdout | 2022-05-19 06:09:41,354 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:09:41,367 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:09:41,368 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:09:41,376 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:09:41,579 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:09:41,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:51.578357+00:00 (in 9.998862 seconds) chunkcleanupworker stdout | 2022-05-19 06:09:41,580 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:51 UTC)" (scheduled at 2022-05-19 06:09:41.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:09:41,580 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:09:41,582 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 9, 41, 580762), True, datetime.datetime(2022, 5, 19, 6, 9, 41, 580762), 0, 'chunk_cleanup/%', 50, 1, 0]) gcworker stdout | 2022-05-19 06:09:41,584 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-19 06:09:41,585 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:11.584428+00:00 (in 29.999322 seconds) gcworker stdout | 2022-05-19 06:09:41,585 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:10:11 UTC)" (scheduled at 2022-05-19 06:09:41.584428+00:00) gcworker stdout | 2022-05-19 06:09:41,585 [62] [DEBUG] [__main__] No GC policies found gcworker stdout | 2022-05-19 06:09:41,586 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:10:11 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:09:41,594 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:09:41,594 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:09:41,594 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:51 UTC)" executed successfully gunicorn-registry stdout | 2022-05-19 06:09:42,540 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-19 06:09:42,548 [234] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '234'} gunicorn-registry stdout | 2022-05-19 06:09:42,554 [238] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '238'} gunicorn-registry stdout | 2022-05-19 06:09:42,560 [232] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '232'} gunicorn-registry stdout | 2022-05-19 06:09:42,567 [241] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '241'} gunicorn-registry stdout | 2022-05-19 06:09:42,572 [240] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '240'} gunicorn-registry stdout | 2022-05-19 06:09:42,576 [236] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '236'} gunicorn-registry stdout | 2022-05-19 06:09:42,590 [239] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '239'} gunicorn-registry stdout | 2022-05-19 06:09:42,597 [237] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '237'} securityworker stdout | 2022-05-19 06:09:43,479 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:09:43,480 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:50.688187+00:00 (in 7.208392 seconds) securityworker stdout | 2022-05-19 06:09:43,480 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:10:13 UTC)" (scheduled at 2022-05-19 06:09:43.478958+00:00) securityworker stdout | 2022-05-19 06:09:43,480 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:09:43,480 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:09:43,483 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:09:43,486 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:09:43,495 [93] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:09:43,496 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:09:43,497 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:10:13 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:09:44,912 [233] [DEBUG] [app] Starting request: urn:request:8e8b2c2f-2aec-4a1d-bc15-0ccd5a52255e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:09:44,913 [231] [DEBUG] [app] Starting request: urn:request:01a4368b-99d4-4a39-8826-2e440c4dad27 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:09:44,915 [233] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:09:44,917 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:09:44,918 [232] [DEBUG] [app] Starting request: urn:request:b55038c6-a0cb-4660-9fe3-c043367f2a91 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:09:44,919 [232] [DEBUG] [app] Ending request: urn:request:b55038c6-a0cb-4660-9fe3-c043367f2a91 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b55038c6-a0cb-4660-9fe3-c043367f2a91', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:09:44,919 [232] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:09:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:09:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.002) gunicorn-web stdout | 2022-05-19 06:09:44,919 [233] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-19 06:09:44,920 [232] [DEBUG] [app] Starting request: urn:request:0ab2607c-382b-4ad4-ae25-cec31499b4cc (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:09:44,921 [232] [DEBUG] [app] Ending request: urn:request:0ab2607c-382b-4ad4-ae25-cec31499b4cc (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0ab2607c-382b-4ad4-ae25-cec31499b4cc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:09:44,922 [232] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:09:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:09:44,922 [233] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost nginx stdout | 127.0.0.1 () - - [19/May/2022:06:09:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.002) gunicorn-web stdout | 2022-05-19 06:09:44,923 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:09:44,924 [233] [DEBUG] [app] Starting request: urn:request:e810a5a3-b1ab-45f8-8760-4f926fd5b87a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:09:44,924 [233] [DEBUG] [app] Ending request: urn:request:e810a5a3-b1ab-45f8-8760-4f926fd5b87a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e810a5a3-b1ab-45f8-8760-4f926fd5b87a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:09:44,925 [233] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:09:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:09:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2022-05-19 06:09:44,925 [233] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:09:44,926 [233] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:09:44,926 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:09:44,929 [230] [DEBUG] [app] Starting request: urn:request:38a2e203-0380-46e6-b877-2a98d1eeb52d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:09:44,930 [230] [DEBUG] [app] Ending request: urn:request:38a2e203-0380-46e6-b877-2a98d1eeb52d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:38a2e203-0380-46e6-b877-2a98d1eeb52d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:09:44,930 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:09:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:09:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2022-05-19 06:09:44,931 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:09:44,932 [231] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:09:44,936 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:09:44,940 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:09:44,944 [233] [DEBUG] [app] Ending request: urn:request:8e8b2c2f-2aec-4a1d-bc15-0ccd5a52255e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8e8b2c2f-2aec-4a1d-bc15-0ccd5a52255e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:09:44,944 [233] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:09:44,945 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:09:44 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:09:44 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.034 118 0.034) gunicorn-web stdout | 2022-05-19 06:09:44,950 [231] [DEBUG] [app] Ending request: urn:request:01a4368b-99d4-4a39-8826-2e440c4dad27 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:01a4368b-99d4-4a39-8826-2e440c4dad27', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:09:44,950 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:09:44,951 [231] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:09:44 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:09:44 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.040 118 0.040) notificationworker stdout | 2022-05-19 06:09:45,405 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:09:45,405 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:48.405016+00:00 (in 2.999628 seconds) notificationworker stdout | 2022-05-19 06:09:45,405 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-19 06:14:45 UTC)" (scheduled at 2022-05-19 06:09:45.404523+00:00) notificationworker stdout | 2022-05-19 06:09:45,407 [73] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 19, 6, 9, 45, 406102), 'notification/%']) notificationworker stdout | 2022-05-19 06:09:45,417 [73] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 19, 6, 9, 45, 406102), True, datetime.datetime(2022, 5, 19, 6, 9, 45, 406102), 0, 'notification/%']) notificationworker stdout | 2022-05-19 06:09:45,421 [73] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 19, 6, 9, 45, 406102), True, datetime.datetime(2022, 5, 19, 6, 9, 45, 406102), 0, 'notification/%', False, datetime.datetime(2022, 5, 19, 6, 9, 45, 406102), 'notification/%']) notificationworker stdout | 2022-05-19 06:09:45,424 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:09:45,424 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-19 06:14:45 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-19 06:09:45,474 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-19 06:09:45,474 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:15.473591+00:00 (in 29.999224 seconds) buildlogsarchiver stdout | 2022-05-19 06:09:45,474 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:10:15 UTC)" (scheduled at 2022-05-19 06:09:45.473591+00:00) buildlogsarchiver stdout | 2022-05-19 06:09:45,475 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 5, 4, 6, 9, 45, 475109), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-19 06:09:45,487 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-19 06:09:45,487 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-19 06:09:45,487 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:10:15 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-19 06:09:46,264 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} namespacegcworker stdout | 2022-05-19 06:09:46,492 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-19 06:09:46,492 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:16.679426+00:00 (in 30.187037 seconds) namespacegcworker stdout | 2022-05-19 06:09:46,492 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:46 UTC)" (scheduled at 2022-05-19 06:09:46.491684+00:00) namespacegcworker stdout | 2022-05-19 06:09:46,493 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-19 06:09:46,494 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 9, 46, 493081), True, datetime.datetime(2022, 5, 19, 6, 9, 46, 493081), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-19 06:09:46,505 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-19 06:09:46,505 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-19 06:09:46,505 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:46 UTC)" executed successfully notificationworker stdout | 2022-05-19 06:09:48,405 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:09:48,406 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:49.297880+00:00 (in 0.892102 seconds) notificationworker stdout | 2022-05-19 06:09:48,406 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:48 UTC)" (scheduled at 2022-05-19 06:09:48.405016+00:00) notificationworker stdout | 2022-05-19 06:09:48,406 [73] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-19 06:09:48,406 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:48 UTC)" executed successfully notificationworker stdout | 2022-05-19 06:09:49,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:09:49,299 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:59.297880+00:00 (in 9.999044 seconds) notificationworker stdout | 2022-05-19 06:09:49,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:59 UTC)" (scheduled at 2022-05-19 06:09:49.297880+00:00) notificationworker stdout | 2022-05-19 06:09:49,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:09:49,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 9, 49, 299538), True, datetime.datetime(2022, 5, 19, 6, 9, 49, 299538), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:09:49,318 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:09:49,318 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:09:49,319 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:09:59 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-19 06:09:50,141 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-19 06:09:50,146 [187] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '187'} gunicorn-secscan stdout | 2022-05-19 06:09:50,152 [188] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '188'} securityworker stdout | 2022-05-19 06:09:50,688 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:09:50,689 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:13.478958+00:00 (in 22.789810 seconds) securityworker stdout | 2022-05-19 06:09:50,689 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:10:20 UTC)" (scheduled at 2022-05-19 06:09:50.688187+00:00) securityworker stdout | 2022-05-19 06:09:50,690 [93] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:09:50,690 [93] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2022-05-19 06:09:50,692 [93] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:09:50,692 [93] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2022-05-19 06:09:50,693 [93] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:09:50,693 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:09:50,693 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:09:50,697 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:09:50,700 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:09:50,710 [93] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:09:50,710 [93] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2022-05-19 06:09:50,711 [93] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2022-05-19 06:09:50,711 [93] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2022-05-19 06:09:50,711 [93] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:09:50,712 [93] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:09:50,712 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:09:50,712 [93] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 46, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 251, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2022-05-19 06:09:50,713 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:10:20 UTC)" executed successfully builder stdout | 2022-05-19 06:09:51,385 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:09:51,400 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:09:51,400 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:09:51,409 [56] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-19 06:09:51,480 [81] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-19 06:09:51,480 [81] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:07.583673+00:00 (in 16.103321 seconds) securityscanningnotificationworker stdout | 2022-05-19 06:09:51,481 [81] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:51 UTC)" (scheduled at 2022-05-19 06:09:51.479280+00:00) securityscanningnotificationworker stdout | 2022-05-19 06:09:51,481 [81] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-19 06:09:51,483 [81] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 9, 51, 481728), True, datetime.datetime(2022, 5, 19, 6, 9, 51, 481728), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-19 06:09:51,497 [81] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-19 06:09:51,497 [81] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-19 06:09:51,498 [81] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:51 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:09:51,579 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:09:51,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:09:59.780399+00:00 (in 8.201053 seconds) chunkcleanupworker stdout | 2022-05-19 06:09:51,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:01 UTC)" (scheduled at 2022-05-19 06:09:51.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:09:51,580 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:09:51,581 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 9, 51, 580110), True, datetime.datetime(2022, 5, 19, 6, 9, 51, 580110), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:09:51,592 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:09:51,592 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:09:51,592 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:01 UTC)" executed successfully gcworker stdout | 2022-05-19 06:09:52,227 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'gcworker.py', 'pid': '62'} namespacegcworker stdout | 2022-05-19 06:09:53,594 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'namespacegcworker.py', 'pid': '71'} servicekey stdout | 2022-05-19 06:09:53,668 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'servicekeyworker.py', 'pid': '94'} repositoryactioncounter stdout | 2022-05-19 06:09:57,879 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositoryactioncounter.py', 'pid': '79'} blobuploadcleanupworker stdout | 2022-05-19 06:09:57,889 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} storagereplication stdout | 2022-05-19 06:09:58,344 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'storagereplication.py', 'pid': '97'} securityworker stdout | 2022-05-19 06:09:58,722 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityworker.py', 'pid': '93'} notificationworker stdout | 2022-05-19 06:09:59,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:09:59,299 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:09.297880+00:00 (in 9.999130 seconds) notificationworker stdout | 2022-05-19 06:09:59,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:09 UTC)" (scheduled at 2022-05-19 06:09:59.297880+00:00) notificationworker stdout | 2022-05-19 06:09:59,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:09:59,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 9, 59, 299481), True, datetime.datetime(2022, 5, 19, 6, 9, 59, 299481), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:09:59,311 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:09:59,312 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:09:59,312 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:09 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:09:59,781 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:09:59,781 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:01.578357+00:00 (in 1.797000 seconds) chunkcleanupworker stdout | 2022-05-19 06:09:59,782 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:59 UTC)" (scheduled at 2022-05-19 06:09:59.780399+00:00) chunkcleanupworker stdout | 2022-05-19 06:09:59,782 [58] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2022-05-19 06:09:59,782 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:10:59 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:09:59,912 [230] [DEBUG] [app] Starting request: urn:request:5724748a-b863-45c7-b508-453df5c00bac (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:09:59,912 [233] [DEBUG] [app] Starting request: urn:request:a20ad260-9567-4810-a4c1-b686ca5599df (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:09:59,914 [233] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:09:59,914 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:09:59,917 [232] [DEBUG] [app] Starting request: urn:request:e8d0f276-ed8d-4866-9b49-f58def0cb2ab (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:09:59,917 [240] [DEBUG] [app] Starting request: urn:request:0c4e2b01-096e-4b2a-9f40-b7025a47552e (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:09:59,918 [240] [DEBUG] [app] Ending request: urn:request:0c4e2b01-096e-4b2a-9f40-b7025a47552e (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0c4e2b01-096e-4b2a-9f40-b7025a47552e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:09:59,918 [232] [DEBUG] [app] Ending request: urn:request:e8d0f276-ed8d-4866-9b49-f58def0cb2ab (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e8d0f276-ed8d-4866-9b49-f58def0cb2ab', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:09:59,918 [240] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:09:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:09:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) nginx stdout | 127.0.0.1 () - - [19/May/2022:06:09:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-registry stdout | 2022-05-19 06:09:59,919 [232] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:09:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:09:59,919 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:09:59,919 [233] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:09:59,921 [233] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:09:59,921 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:09:59,923 [230] [DEBUG] [app] Starting request: urn:request:eb0b75f5-88fb-43e8-9413-5e2a128d4f90 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:09:59,923 [235] [DEBUG] [app] Starting request: urn:request:d9515c9d-83ee-4788-bb2b-666400f477a2 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:09:59,924 [235] [DEBUG] [app] Ending request: urn:request:d9515c9d-83ee-4788-bb2b-666400f477a2 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d9515c9d-83ee-4788-bb2b-666400f477a2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:09:59,924 [230] [DEBUG] [app] Ending request: urn:request:eb0b75f5-88fb-43e8-9413-5e2a128d4f90 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:eb0b75f5-88fb-43e8-9413-5e2a128d4f90', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:09:59,924 [235] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:09:59 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:09:59,924 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:09:59 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:09:59 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [19/May/2022:06:09:59 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2022-05-19 06:09:59,925 [233] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:09:59,925 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:09:59,925 [230] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:09:59,926 [233] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:09:59,933 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:09:59,933 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:09:59,942 [233] [DEBUG] [app] Ending request: urn:request:a20ad260-9567-4810-a4c1-b686ca5599df (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a20ad260-9567-4810-a4c1-b686ca5599df', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:09:59,942 [230] [DEBUG] [app] Ending request: urn:request:5724748a-b863-45c7-b508-453df5c00bac (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5724748a-b863-45c7-b508-453df5c00bac', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:09:59,943 [233] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:09:59,943 [230] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:09:59,943 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:09:59 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-19 06:09:59,943 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:09:59 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:09:59 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.032 118 0.032) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:09:59 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.033 118 0.033) buildlogsarchiver stdout | 2022-05-19 06:10:00,491 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} queuecleanupworker stdout | 2022-05-19 06:10:00,666 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'queuecleanupworker.py', 'pid': '75'} globalpromstats stdout | 2022-05-19 06:10:00,878 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'globalpromstats.py', 'pid': '63'} teamsyncworker stdout | 2022-05-19 06:10:01,162 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'teamsyncworker.py', 'pid': '101'} builder stdout | 2022-05-19 06:10:01,419 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:10:01,432 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:10:01,433 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:10:01,442 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:10:01,579 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:10:01,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:11.578357+00:00 (in 9.999003 seconds) chunkcleanupworker stdout | 2022-05-19 06:10:01,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:11 UTC)" (scheduled at 2022-05-19 06:10:01.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:10:01,580 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:10:01,581 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 10, 1, 580151), True, datetime.datetime(2022, 5, 19, 6, 10, 1, 580151), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:10:01,595 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:10:01,595 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:10:01,595 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:11 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:10:02,682 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} expiredappspecifictokenworker stdout | 2022-05-19 06:10:02,987 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} notificationworker stdout | 2022-05-19 06:10:03,155 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'notificationworker.py', 'pid': '73'} builder stdout | 2022-05-19 06:10:03,194 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'builder.py', 'pid': '56'} repositorygcworker stdout | 2022-05-19 06:10:03,458 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositorygcworker.py', 'pid': '80'} manifestbackfillworker stdout | 2022-05-19 06:10:05,868 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} securityscanningnotificationworker stdout | 2022-05-19 06:10:07,003 [81] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityscanningnotificationworker.py', 'pid': '81'} logrotateworker stdout | 2022-05-19 06:10:07,405 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'logrotateworker.py', 'pid': '68'} securityscanningnotificationworker stdout | 2022-05-19 06:10:07,584 [81] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-19 06:10:07,585 [81] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:51.479280+00:00 (in 43.894530 seconds) securityscanningnotificationworker stdout | 2022-05-19 06:10:07,585 [81] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:07 UTC)" (scheduled at 2022-05-19 06:10:07.583673+00:00) securityscanningnotificationworker stdout | 2022-05-19 06:10:07,585 [81] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-19 06:10:07,585 [81] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:07 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:10:08,015 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '231'} gunicorn-web stdout | 2022-05-19 06:10:08,020 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '230'} gunicorn-web stdout | 2022-05-19 06:10:08,028 [235] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '235'} gunicorn-web stdout | 2022-05-19 06:10:08,033 [233] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '233'} gunicorn-web stdout | 2022-05-19 06:10:08,042 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-19 06:10:09,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:10:09,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:19.297880+00:00 (in 9.999173 seconds) notificationworker stdout | 2022-05-19 06:10:09,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:19 UTC)" (scheduled at 2022-05-19 06:10:09.297880+00:00) notificationworker stdout | 2022-05-19 06:10:09,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:10:09,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 10, 9, 299389), True, datetime.datetime(2022, 5, 19, 6, 10, 9, 299389), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:10:09,320 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:10:09,320 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:10:09,320 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:19 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-19 06:10:09,793 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-19 06:10:09,794 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:36.193076+00:00 (in 26.399142 seconds) exportactionlogsworker stdout | 2022-05-19 06:10:09,794 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:09 UTC)" (scheduled at 2022-05-19 06:10:09.792901+00:00) exportactionlogsworker stdout | 2022-05-19 06:10:09,794 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-19 06:10:09,796 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 10, 9, 794909), True, datetime.datetime(2022, 5, 19, 6, 10, 9, 794909), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-19 06:10:09,809 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-19 06:10:09,809 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-19 06:10:09,810 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:09 UTC)" executed successfully builder stdout | 2022-05-19 06:10:11,445 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:10:11,458 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:10:11,458 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:10:11,468 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:10:11,579 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:10:11,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:21.578357+00:00 (in 9.999060 seconds) chunkcleanupworker stdout | 2022-05-19 06:10:11,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:21 UTC)" (scheduled at 2022-05-19 06:10:11.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:10:11,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:10:11,581 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 10, 11, 579980), True, datetime.datetime(2022, 5, 19, 6, 10, 11, 579980), 0, 'chunk_cleanup/%', 50, 1, 0]) gcworker stdout | 2022-05-19 06:10:11,585 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-19 06:10:11,585 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:41.584428+00:00 (in 29.999048 seconds) gcworker stdout | 2022-05-19 06:10:11,586 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:10:41 UTC)" (scheduled at 2022-05-19 06:10:11.584428+00:00) gcworker stdout | 2022-05-19 06:10:11,586 [62] [DEBUG] [__main__] No GC policies found gcworker stdout | 2022-05-19 06:10:11,586 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:10:41 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:10:11,592 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:10:11,592 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:10:11,592 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:21 UTC)" executed successfully gunicorn-registry stdout | 2022-05-19 06:10:12,559 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-19 06:10:12,565 [234] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '234'} gunicorn-registry stdout | 2022-05-19 06:10:12,571 [238] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '238'} gunicorn-registry stdout | 2022-05-19 06:10:12,576 [232] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '232'} gunicorn-registry stdout | 2022-05-19 06:10:12,582 [241] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '241'} gunicorn-registry stdout | 2022-05-19 06:10:12,587 [240] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '240'} gunicorn-registry stdout | 2022-05-19 06:10:12,593 [236] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '236'} gunicorn-registry stdout | 2022-05-19 06:10:12,606 [239] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '239'} gunicorn-registry stdout | 2022-05-19 06:10:12,612 [237] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '237'} securityworker stdout | 2022-05-19 06:10:13,479 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:10:13,480 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:20.688187+00:00 (in 7.208213 seconds) securityworker stdout | 2022-05-19 06:10:13,480 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:10:43 UTC)" (scheduled at 2022-05-19 06:10:13.478958+00:00) securityworker stdout | 2022-05-19 06:10:13,481 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:10:13,481 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:10:13,484 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:10:13,487 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:10:13,497 [93] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:10:13,500 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:10:13,500 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:10:43 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:10:14,912 [231] [DEBUG] [app] Starting request: urn:request:be41d9ab-1b2f-470a-bafc-0327162bdd23 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:10:14,914 [230] [DEBUG] [app] Starting request: urn:request:5de9b487-f25c-4f31-9577-f27dc29e7062 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:10:14,915 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:10:14,916 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:10:14,918 [232] [DEBUG] [app] Starting request: urn:request:69a66fa1-636a-427d-8543-e038647971f7 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:10:14,919 [232] [DEBUG] [app] Ending request: urn:request:69a66fa1-636a-427d-8543-e038647971f7 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:69a66fa1-636a-427d-8543-e038647971f7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:10:14,919 [232] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:10:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2022-05-19 06:10:14,919 [239] [DEBUG] [app] Starting request: urn:request:531468fc-95c7-4054-a4a2-264059b6ed5a (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:10:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2022-05-19 06:10:14,920 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-19 06:10:14,921 [239] [DEBUG] [app] Ending request: urn:request:531468fc-95c7-4054-a4a2-264059b6ed5a (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:531468fc-95c7-4054-a4a2-264059b6ed5a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:10:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.006 162 0.007) gunicorn-web stdout | 2022-05-19 06:10:14,922 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:10:14,923 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-19 06:10:14,922 [239] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:10:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:10:14,925 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:10:14,925 [235] [DEBUG] [app] Starting request: urn:request:af67db92-f7f9-4a51-9675-6ffaff65857b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:10:14,926 [235] [DEBUG] [app] Ending request: urn:request:af67db92-f7f9-4a51-9675-6ffaff65857b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:af67db92-f7f9-4a51-9675-6ffaff65857b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:10:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2022-05-19 06:10:14,927 [235] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:10:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:10:14,927 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:10:14,927 [231] [DEBUG] [app] Starting request: urn:request:aa8f723e-ec1c-4829-bf73-a6da805a59b5 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:10:14,928 [230] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:10:14,928 [231] [DEBUG] [app] Ending request: urn:request:aa8f723e-ec1c-4829-bf73-a6da805a59b5 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:aa8f723e-ec1c-4829-bf73-a6da805a59b5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:10:14,928 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:10:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:10:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2022-05-19 06:10:14,929 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:10:14,930 [231] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:10:14,935 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:10:14,937 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:10:14,944 [230] [DEBUG] [app] Ending request: urn:request:5de9b487-f25c-4f31-9577-f27dc29e7062 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5de9b487-f25c-4f31-9577-f27dc29e7062', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:10:14,944 [230] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:10:14 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.033 118 0.032) gunicorn-web stdout | 2022-05-19 06:10:14,945 [231] [DEBUG] [app] Ending request: urn:request:be41d9ab-1b2f-470a-bafc-0327162bdd23 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:be41d9ab-1b2f-470a-bafc-0327162bdd23', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:10:14,945 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:10:14,945 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:10:14 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:10:14 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.036 118 0.035) gunicorn-web stdout | 2022-05-19 06:10:14,946 [231] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:10:14 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-19 06:10:15,228 [233] [DEBUG] [app] Starting request: urn:request:069c5e99-2dc1-424b-a37f-6a5ba85a8a91 (/) {'X-Forwarded-For': '10.131.0.63'} gunicorn-web stdout | 2022-05-19 06:10:15,249 [233] [DEBUG] [endpoints.api.discovery] Unable to find method for POST in class gunicorn-web stdout | 2022-05-19 06:10:15,280 [233] [DEBUG] [app] Ending request: urn:request:069c5e99-2dc1-424b-a37f-6a5ba85a8a91 (/) {'endpoint': 'web.index', 'request_id': 'urn:request:069c5e99-2dc1-424b-a37f-6a5ba85a8a91', 'remote_addr': '10.131.0.63', 'http_method': 'GET', 'original_url': 'http://quayregistry-quay-app/', 'path': '/', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'curl/7.61.1'} gunicorn-web stdout | 2022-05-19 06:10:15,281 [233] [INFO] [gunicorn.access] 10.131.0.63 - - [19/May/2022:06:10:15 +0000] "GET / HTTP/1.0" 200 84286 "-" "curl/7.61.1" nginx stdout | 10.131.0.63 () - - [19/May/2022:06:10:15 +0000] "GET / HTTP/1.1" 200 84286 "-" "curl/7.61.1" (0.057 85 0.055) buildlogsarchiver stdout | 2022-05-19 06:10:15,474 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-19 06:10:15,475 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:45.473591+00:00 (in 29.999079 seconds) buildlogsarchiver stdout | 2022-05-19 06:10:15,475 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:10:45 UTC)" (scheduled at 2022-05-19 06:10:15.473591+00:00) buildlogsarchiver stdout | 2022-05-19 06:10:15,476 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 5, 4, 6, 10, 15, 475614), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-19 06:10:15,487 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-19 06:10:15,487 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-19 06:10:15,488 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:10:45 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-19 06:10:16,289 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} namespacegcworker stdout | 2022-05-19 06:10:16,680 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-19 06:10:16,680 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:46.491684+00:00 (in 29.811370 seconds) namespacegcworker stdout | 2022-05-19 06:10:16,681 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:16 UTC)" (scheduled at 2022-05-19 06:10:16.679426+00:00) namespacegcworker stdout | 2022-05-19 06:10:16,681 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-19 06:10:16,681 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:16 UTC)" executed successfully repositorygcworker stdout | 2022-05-19 06:10:19,097 [80] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-19 06:10:19,098 [80] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:29.976033+00:00 (in 10.878320 seconds) repositorygcworker stdout | 2022-05-19 06:10:19,098 [80] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-19 06:15:19 UTC)" (scheduled at 2022-05-19 06:10:19.096808+00:00) repositorygcworker stdout | 2022-05-19 06:10:19,099 [80] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 19, 6, 10, 19, 98540), 'repositorygc/%']) repositorygcworker stdout | 2022-05-19 06:10:19,112 [80] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 19, 6, 10, 19, 98540), True, datetime.datetime(2022, 5, 19, 6, 10, 19, 98540), 0, 'repositorygc/%']) repositorygcworker stdout | 2022-05-19 06:10:19,116 [80] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 19, 6, 10, 19, 98540), True, datetime.datetime(2022, 5, 19, 6, 10, 19, 98540), 0, 'repositorygc/%', False, datetime.datetime(2022, 5, 19, 6, 10, 19, 98540), 'repositorygc/%']) repositorygcworker stdout | 2022-05-19 06:10:19,119 [80] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-19 06:10:19,119 [80] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-19 06:15:19 UTC)" executed successfully notificationworker stdout | 2022-05-19 06:10:19,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:10:19,299 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:29.297880+00:00 (in 9.999162 seconds) notificationworker stdout | 2022-05-19 06:10:19,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:29 UTC)" (scheduled at 2022-05-19 06:10:19.297880+00:00) notificationworker stdout | 2022-05-19 06:10:19,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:10:19,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 10, 19, 299517), True, datetime.datetime(2022, 5, 19, 6, 10, 19, 299517), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:10:19,320 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:10:19,320 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:10:19,320 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:29 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-19 06:10:20,157 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-19 06:10:20,164 [187] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '187'} gunicorn-secscan stdout | 2022-05-19 06:10:20,170 [188] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '188'} securityworker stdout | 2022-05-19 06:10:20,688 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:10:20,689 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:43.478958+00:00 (in 22.789841 seconds) securityworker stdout | 2022-05-19 06:10:20,689 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:10:50 UTC)" (scheduled at 2022-05-19 06:10:20.688187+00:00) securityworker stdout | 2022-05-19 06:10:20,689 [93] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:10:20,689 [93] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2022-05-19 06:10:20,691 [93] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:10:20,691 [93] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2022-05-19 06:10:20,691 [93] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:10:20,692 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:10:20,692 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:10:20,695 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:10:20,697 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:10:20,707 [93] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:10:20,708 [93] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2022-05-19 06:10:20,708 [93] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2022-05-19 06:10:20,708 [93] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2022-05-19 06:10:20,708 [93] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:10:20,709 [93] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:10:20,709 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:10:20,709 [93] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 46, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 251, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2022-05-19 06:10:20,710 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:10:50 UTC)" executed successfully builder stdout | 2022-05-19 06:10:21,477 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:10:21,491 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:10:21,491 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:10:21,502 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:10:21,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:10:21,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:31.578357+00:00 (in 9.999192 seconds) chunkcleanupworker stdout | 2022-05-19 06:10:21,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:31 UTC)" (scheduled at 2022-05-19 06:10:21.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:10:21,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:10:21,581 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 10, 21, 579989), True, datetime.datetime(2022, 5, 19, 6, 10, 21, 579989), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:10:21,592 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:10:21,592 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:10:21,593 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:31 UTC)" executed successfully gcworker stdout | 2022-05-19 06:10:22,268 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'gcworker.py', 'pid': '62'} namespacegcworker stdout | 2022-05-19 06:10:23,619 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'namespacegcworker.py', 'pid': '71'} servicekey stdout | 2022-05-19 06:10:23,691 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'servicekeyworker.py', 'pid': '94'} repositoryactioncounter stdout | 2022-05-19 06:10:27,920 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositoryactioncounter.py', 'pid': '79'} blobuploadcleanupworker stdout | 2022-05-19 06:10:27,926 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} storagereplication stdout | 2022-05-19 06:10:28,384 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'storagereplication.py', 'pid': '97'} securityworker stdout | 2022-05-19 06:10:28,766 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityworker.py', 'pid': '93'} notificationworker stdout | 2022-05-19 06:10:29,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:10:29,299 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:39.297880+00:00 (in 9.999071 seconds) notificationworker stdout | 2022-05-19 06:10:29,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:39 UTC)" (scheduled at 2022-05-19 06:10:29.297880+00:00) notificationworker stdout | 2022-05-19 06:10:29,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:10:29,301 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 10, 29, 299882), True, datetime.datetime(2022, 5, 19, 6, 10, 29, 299882), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:10:29,312 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:10:29,313 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:10:29,313 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:39 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:10:29,912 [230] [DEBUG] [app] Starting request: urn:request:275472de-cf40-4086-8356-4286e732ed75 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:10:29,912 [235] [DEBUG] [app] Starting request: urn:request:0cb3f922-4b5e-4b7d-934f-c3f1c70c2d51 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:10:29,914 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:10:29,915 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:10:29,917 [232] [DEBUG] [app] Starting request: urn:request:1e8788d4-449a-434c-881c-2c86efcf5805 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:10:29,918 [232] [DEBUG] [app] Ending request: urn:request:1e8788d4-449a-434c-881c-2c86efcf5805 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:1e8788d4-449a-434c-881c-2c86efcf5805', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:10:29,919 [232] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:10:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:10:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.002) gunicorn-web stdout | 2022-05-19 06:10:29,919 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-19 06:10:29,920 [237] [DEBUG] [app] Starting request: urn:request:42c8fd7b-9b1e-4515-9d69-11365b26eda6 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:10:29,922 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:10:29,922 [237] [DEBUG] [app] Ending request: urn:request:42c8fd7b-9b1e-4515-9d69-11365b26eda6 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:42c8fd7b-9b1e-4515-9d69-11365b26eda6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:10:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.008 162 0.008) gunicorn-registry stdout | 2022-05-19 06:10:29,924 [237] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:10:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:10:29,924 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:10:29,925 [233] [DEBUG] [app] Starting request: urn:request:5ae72169-e565-4f59-a5bc-c9598dbd6c9b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:10:29,926 [233] [DEBUG] [app] Ending request: urn:request:5ae72169-e565-4f59-a5bc-c9598dbd6c9b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:5ae72169-e565-4f59-a5bc-c9598dbd6c9b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:10:29,926 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:10:29,926 [233] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:10:29 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:10:29 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2022-05-19 06:10:29,926 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:10:29,927 [235] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:10:29,928 [233] [DEBUG] [app] Starting request: urn:request:164dbf88-82cb-4efc-80e2-6e2b9c11b1a0 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:10:29,929 [233] [DEBUG] [app] Ending request: urn:request:164dbf88-82cb-4efc-80e2-6e2b9c11b1a0 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:164dbf88-82cb-4efc-80e2-6e2b9c11b1a0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:10:29,929 [233] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:10:29 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:10:29 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2022-05-19 06:10:29,929 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:10:29,930 [230] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:10:29,935 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:10:29,937 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:10:29,944 [235] [DEBUG] [app] Ending request: urn:request:0cb3f922-4b5e-4b7d-934f-c3f1c70c2d51 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0cb3f922-4b5e-4b7d-934f-c3f1c70c2d51', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:10:29,945 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:10:29,945 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:10:29 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:10:29 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.034 118 0.035) gunicorn-web stdout | 2022-05-19 06:10:29,947 [230] [DEBUG] [app] Ending request: urn:request:275472de-cf40-4086-8356-4286e732ed75 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:275472de-cf40-4086-8356-4286e732ed75', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:10:29,947 [230] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:10:29,948 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:10:29 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:10:29 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.037 118 0.037) repositorygcworker stdout | 2022-05-19 06:10:29,976 [80] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-19 06:10:29,977 [80] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:31.173528+00:00 (in 1.196531 seconds) repositorygcworker stdout | 2022-05-19 06:10:29,977 [80] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:29 UTC)" (scheduled at 2022-05-19 06:10:29.976033+00:00) repositorygcworker stdout | 2022-05-19 06:10:29,977 [80] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-19 06:10:29,979 [80] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 10, 29, 977877), True, datetime.datetime(2022, 5, 19, 6, 10, 29, 977877), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-19 06:10:29,991 [80] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-19 06:10:29,991 [80] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-19 06:10:29,991 [80] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:29 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-19 06:10:30,533 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} queuecleanupworker stdout | 2022-05-19 06:10:30,702 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'queuecleanupworker.py', 'pid': '75'} gunicorn-web stdout | 2022-05-19 06:10:30,849 [235] [DEBUG] [app] Starting request: urn:request:32d11aca-5480-47e1-a539-568eb86f6f52 (/) {'X-Forwarded-For': '10.128.2.23'} gunicorn-web stdout | 2022-05-19 06:10:30,863 [235] [DEBUG] [endpoints.api.discovery] Unable to find method for POST in class gunicorn-web stdout | 2022-05-19 06:10:30,895 [235] [DEBUG] [app] Ending request: urn:request:32d11aca-5480-47e1-a539-568eb86f6f52 (/) {'endpoint': 'web.index', 'request_id': 'urn:request:32d11aca-5480-47e1-a539-568eb86f6f52', 'remote_addr': '10.128.2.23', 'http_method': 'GET', 'original_url': 'http://quayregistry-quay-app/', 'path': '/', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'curl/7.61.1'} gunicorn-web stdout | 2022-05-19 06:10:30,896 [235] [INFO] [gunicorn.access] 10.128.2.23 - - [19/May/2022:06:10:30 +0000] "GET / HTTP/1.0" 200 84286 "-" "curl/7.61.1" nginx stdout | 10.128.2.23 () - - [19/May/2022:06:10:30 +0000] "GET / HTTP/1.1" 200 84286 "-" "curl/7.61.1" (0.051 85 0.050) globalpromstats stdout | 2022-05-19 06:10:30,921 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'globalpromstats.py', 'pid': '63'} repositorygcworker stdout | 2022-05-19 06:10:31,174 [80] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-19 06:10:31,174 [80] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:29.976033+00:00 (in 58.801710 seconds) repositorygcworker stdout | 2022-05-19 06:10:31,174 [80] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:31 UTC)" (scheduled at 2022-05-19 06:10:31.173528+00:00) repositorygcworker stdout | 2022-05-19 06:10:31,174 [80] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-19 06:10:31,175 [80] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:31 UTC)" executed successfully teamsyncworker stdout | 2022-05-19 06:10:31,198 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'teamsyncworker.py', 'pid': '101'} builder stdout | 2022-05-19 06:10:31,512 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:10:31,526 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:10:31,526 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:10:31,535 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:10:31,579 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:10:31,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:41.578357+00:00 (in 9.999147 seconds) chunkcleanupworker stdout | 2022-05-19 06:10:31,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:41 UTC)" (scheduled at 2022-05-19 06:10:31.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:10:31,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:10:31,581 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 10, 31, 579884), True, datetime.datetime(2022, 5, 19, 6, 10, 31, 579884), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:10:31,592 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:10:31,592 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:10:31,592 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:41 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:10:32,722 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} expiredappspecifictokenworker stdout | 2022-05-19 06:10:33,021 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} notificationworker stdout | 2022-05-19 06:10:33,196 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'notificationworker.py', 'pid': '73'} builder stdout | 2022-05-19 06:10:33,228 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'builder.py', 'pid': '56'} repositorygcworker stdout | 2022-05-19 06:10:33,498 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositorygcworker.py', 'pid': '80'} manifestbackfillworker stdout | 2022-05-19 06:10:35,891 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} exportactionlogsworker stdout | 2022-05-19 06:10:36,193 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-19 06:10:36,194 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:58.192705+00:00 (in 21.998635 seconds) exportactionlogsworker stdout | 2022-05-19 06:10:36,194 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:36 UTC)" (scheduled at 2022-05-19 06:10:36.193076+00:00) exportactionlogsworker stdout | 2022-05-19 06:10:36,194 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-19 06:10:36,194 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:36 UTC)" executed successfully securityscanningnotificationworker stdout | 2022-05-19 06:10:37,018 [81] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityscanningnotificationworker.py', 'pid': '81'} logrotateworker stdout | 2022-05-19 06:10:37,426 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'logrotateworker.py', 'pid': '68'} gunicorn-web stdout | 2022-05-19 06:10:38,029 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '231'} gunicorn-web stdout | 2022-05-19 06:10:38,037 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '230'} gunicorn-web stdout | 2022-05-19 06:10:38,047 [235] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '235'} gunicorn-web stdout | 2022-05-19 06:10:38,058 [233] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '233'} gunicorn-web stdout | 2022-05-19 06:10:38,067 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-19 06:10:39,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:10:39,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:48.405016+00:00 (in 9.106274 seconds) notificationworker stdout | 2022-05-19 06:10:39,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:49 UTC)" (scheduled at 2022-05-19 06:10:39.297880+00:00) notificationworker stdout | 2022-05-19 06:10:39,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:10:39,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 10, 39, 299409), True, datetime.datetime(2022, 5, 19, 6, 10, 39, 299409), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:10:39,311 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:10:39,311 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:10:39,311 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:49 UTC)" executed successfully builder stdout | 2022-05-19 06:10:41,542 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:10:41,557 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:10:41,557 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:10:41,568 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:10:41,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:10:41,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:51.578357+00:00 (in 9.999328 seconds) chunkcleanupworker stdout | 2022-05-19 06:10:41,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:51 UTC)" (scheduled at 2022-05-19 06:10:41.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:10:41,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:10:41,580 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 10, 41, 579734), True, datetime.datetime(2022, 5, 19, 6, 10, 41, 579734), 0, 'chunk_cleanup/%', 50, 1, 0]) gcworker stdout | 2022-05-19 06:10:41,585 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-19 06:10:41,586 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:11.584428+00:00 (in 29.998798 seconds) gcworker stdout | 2022-05-19 06:10:41,586 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:11:11 UTC)" (scheduled at 2022-05-19 06:10:41.584428+00:00) gcworker stdout | 2022-05-19 06:10:41,586 [62] [DEBUG] [__main__] No GC policies found gcworker stdout | 2022-05-19 06:10:41,586 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:11:11 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:10:41,598 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:10:41,598 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:10:41,598 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:51 UTC)" executed successfully gunicorn-registry stdout | 2022-05-19 06:10:42,574 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-19 06:10:42,581 [234] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '234'} gunicorn-registry stdout | 2022-05-19 06:10:42,590 [238] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '238'} gunicorn-registry stdout | 2022-05-19 06:10:42,598 [232] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '232'} gunicorn-registry stdout | 2022-05-19 06:10:42,604 [241] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '241'} gunicorn-registry stdout | 2022-05-19 06:10:42,611 [240] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '240'} gunicorn-registry stdout | 2022-05-19 06:10:42,617 [236] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '236'} gunicorn-registry stdout | 2022-05-19 06:10:42,624 [239] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '239'} gunicorn-registry stdout | 2022-05-19 06:10:42,629 [237] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '237'} securityworker stdout | 2022-05-19 06:10:43,479 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:10:43,480 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:50.688187+00:00 (in 7.208343 seconds) securityworker stdout | 2022-05-19 06:10:43,480 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:11:13 UTC)" (scheduled at 2022-05-19 06:10:43.478958+00:00) securityworker stdout | 2022-05-19 06:10:43,481 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:10:43,481 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:10:43,484 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:10:43,486 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:10:43,496 [93] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:10:43,498 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:10:43,498 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:11:13 UTC)" executed successfully servicekey stdout | 2022-05-19 06:10:43,973 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run servicekey stdout | 2022-05-19 06:10:43,974 [94] [INFO] [apscheduler.executors.default] Running job "ServiceKeyWorker._refresh_service_key (trigger: interval[0:55:00], next run at: 2022-05-19 06:10:43 UTC)" (scheduled at 2022-05-19 06:10:43.972747+00:00) servicekey stdout | 2022-05-19 06:10:43,975 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 07:05:43.972747+00:00 (in 3299.999429 seconds) servicekey stdout | 2022-05-19 06:10:43,975 [94] [DEBUG] [__main__] Starting automatic refresh of service key _IhqQsDfhyqGgRlk0eMHv_QJut6MqBRSuqyYsXKAqds to new expiration 2022-05-19 08:10:43.974750 servicekey stdout | 2022-05-19 06:10:43,978 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE (("t1"."kid" = %s) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s))) LIMIT %s OFFSET %s', ['_IhqQsDfhyqGgRlk0eMHv_QJut6MqBRSuqyYsXKAqds', datetime.datetime(2022, 5, 12, 6, 10, 43, 976301), None, 1, 0]) servicekey stdout | 2022-05-19 06:10:43,991 [94] [DEBUG] [peewee] ('UPDATE "servicekey" SET "name" = %s, "kid" = %s, "service" = %s, "jwk" = %s, "metadata" = %s, "created_date" = %s, "expiration_date" = %s, "rotation_duration" = %s, "approval_id" = %s WHERE ("servicekey"."id" = %s)', ['https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com:443', '_IhqQsDfhyqGgRlk0eMHv_QJut6MqBRSuqyYsXKAqds', 'quay', '{"n": "2jq3Ca4q_kzf9jW5iu08hQ_USujsxjnRZu5c_KELSndf6wj7cMikrTAYAfaxMm7mSM24TuASE18J5EeZeoTpZDFQX_4fNF3DpvzWu7vg2LLdVhzBPOBFsc8kf-NuRgSbvZaQJSQOZAMEOkptyDcSWHm4cVLCyqc9upqakKEZQd4nrVnSIgLloyXrMop_xvw0AwoCz-gO50NNK8jscu7gwFtrgFeXiaj9aSud2-9mno5exdMseVhlgKKw8tu7BiyTqWMfKPaXdd_4uWb5Y7wzqqw9h5VX4RGPttiCvF2qg620-M9yBPvORDcU3ec7Kfdxy2407Tt1M8_8ssYnLcOv2w", "e": "AQAB", "kty": "RSA", "kid": "_IhqQsDfhyqGgRlk0eMHv_QJut6MqBRSuqyYsXKAqds"}', '{"created_by": "CLI tool"}', datetime.datetime(2022, 5, 19, 6, 8, 7, 378224), datetime.datetime(2022, 5, 19, 8, 10, 43, 974750), None, 1, 1]) servicekey stdout | 2022-05-19 06:10:43,995 [94] [DEBUG] [__main__] Finished automatic refresh of service key _IhqQsDfhyqGgRlk0eMHv_QJut6MqBRSuqyYsXKAqds with new expiration 2022-05-19 08:10:43.974750 servicekey stdout | 2022-05-19 06:10:43,996 [94] [DEBUG] [data.database] Disconnecting from database. servicekey stdout | 2022-05-19 06:10:43,996 [94] [INFO] [apscheduler.executors.default] Job "ServiceKeyWorker._refresh_service_key (trigger: interval[0:55:00], next run at: 2022-05-19 07:05:43 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:10:44,912 [233] [DEBUG] [app] Starting request: urn:request:a7513f72-0b18-491e-bdcf-bfa1e28292d2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:10:44,912 [230] [DEBUG] [app] Starting request: urn:request:9238f674-398d-4725-9072-afc246e4102c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:10:44,914 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:10:44,914 [233] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:10:44,917 [238] [DEBUG] [app] Starting request: urn:request:3751456a-9283-4cfc-bb71-c5c28b4700ee (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:10:44,918 [238] [DEBUG] [app] Ending request: urn:request:3751456a-9283-4cfc-bb71-c5c28b4700ee (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:3751456a-9283-4cfc-bb71-c5c28b4700ee', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:10:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.003) gunicorn-web stdout | 2022-05-19 06:10:44,919 [233] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-19 06:10:44,920 [238] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:10:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2022-05-19 06:10:44,920 [240] [DEBUG] [app] Starting request: urn:request:31ed4f6d-4cdd-4a52-8c77-51b0ee8b742f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:10:44,921 [240] [DEBUG] [app] Ending request: urn:request:31ed4f6d-4cdd-4a52-8c77-51b0ee8b742f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:31ed4f6d-4cdd-4a52-8c77-51b0ee8b742f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:10:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.004) gunicorn-registry stdout | 2022-05-19 06:10:44,921 [240] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:10:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:10:44,921 [233] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:10:44,922 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:10:44,924 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:10:44,924 [233] [DEBUG] [app] Starting request: urn:request:0245f9a6-27c9-4ccc-a112-0b637fc06975 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:10:44,924 [233] [DEBUG] [app] Ending request: urn:request:0245f9a6-27c9-4ccc-a112-0b637fc06975 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0245f9a6-27c9-4ccc-a112-0b637fc06975', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:10:44,925 [233] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:10:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:10:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2022-05-19 06:10:44,925 [233] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:10:44,926 [230] [DEBUG] [app] Starting request: urn:request:d899cfc4-4c9a-49c8-acad-9fc415f754b3 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:10:44,926 [233] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:10:44,927 [230] [DEBUG] [app] Ending request: urn:request:d899cfc4-4c9a-49c8-acad-9fc415f754b3 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d899cfc4-4c9a-49c8-acad-9fc415f754b3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:10:44,927 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:10:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:10:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2022-05-19 06:10:44,928 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:10:44,929 [230] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:10:44,934 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:10:44,935 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:10:44,942 [233] [DEBUG] [app] Ending request: urn:request:a7513f72-0b18-491e-bdcf-bfa1e28292d2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a7513f72-0b18-491e-bdcf-bfa1e28292d2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:10:44,942 [233] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:10:44,943 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:10:44 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:10:44 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.031 118 0.032) gunicorn-web stdout | 2022-05-19 06:10:44,945 [230] [DEBUG] [app] Ending request: urn:request:9238f674-398d-4725-9072-afc246e4102c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9238f674-398d-4725-9072-afc246e4102c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:10:44,946 [230] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:10:44 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.035 118 0.035) gunicorn-web stdout | 2022-05-19 06:10:44,947 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:10:44 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" buildlogsarchiver stdout | 2022-05-19 06:10:45,474 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-19 06:10:45,475 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:15.473591+00:00 (in 29.998801 seconds) buildlogsarchiver stdout | 2022-05-19 06:10:45,475 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:11:15 UTC)" (scheduled at 2022-05-19 06:10:45.473591+00:00) buildlogsarchiver stdout | 2022-05-19 06:10:45,476 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 5, 4, 6, 10, 45, 475524), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-19 06:10:45,487 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-19 06:10:45,488 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-19 06:10:45,488 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:11:15 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-19 06:10:46,326 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} namespacegcworker stdout | 2022-05-19 06:10:46,492 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-19 06:10:46,493 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:16.679426+00:00 (in 30.186576 seconds) namespacegcworker stdout | 2022-05-19 06:10:46,493 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:46 UTC)" (scheduled at 2022-05-19 06:10:46.491684+00:00) namespacegcworker stdout | 2022-05-19 06:10:46,493 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-19 06:10:46,494 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 10, 46, 493774), True, datetime.datetime(2022, 5, 19, 6, 10, 46, 493774), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-19 06:10:46,506 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-19 06:10:46,506 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-19 06:10:46,506 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:46 UTC)" executed successfully notificationworker stdout | 2022-05-19 06:10:48,405 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:10:48,406 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:49.297880+00:00 (in 0.892023 seconds) notificationworker stdout | 2022-05-19 06:10:48,406 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:48 UTC)" (scheduled at 2022-05-19 06:10:48.405016+00:00) notificationworker stdout | 2022-05-19 06:10:48,406 [73] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-19 06:10:48,406 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:48 UTC)" executed successfully notificationworker stdout | 2022-05-19 06:10:49,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:10:49,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:59.297880+00:00 (in 9.999200 seconds) notificationworker stdout | 2022-05-19 06:10:49,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:59 UTC)" (scheduled at 2022-05-19 06:10:49.297880+00:00) notificationworker stdout | 2022-05-19 06:10:49,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:10:49,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 10, 49, 299346), True, datetime.datetime(2022, 5, 19, 6, 10, 49, 299346), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:10:49,312 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:10:49,312 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:10:49,312 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:10:59 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-19 06:10:50,172 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-19 06:10:50,180 [187] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '187'} gunicorn-secscan stdout | 2022-05-19 06:10:50,188 [188] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '188'} securityworker stdout | 2022-05-19 06:10:50,688 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:10:50,689 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:13.478958+00:00 (in 22.789911 seconds) securityworker stdout | 2022-05-19 06:10:50,689 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:11:20 UTC)" (scheduled at 2022-05-19 06:10:50.688187+00:00) securityworker stdout | 2022-05-19 06:10:50,689 [93] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:10:50,689 [93] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2022-05-19 06:10:50,691 [93] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:10:50,691 [93] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2022-05-19 06:10:50,691 [93] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:10:50,691 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:10:50,691 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:10:50,694 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:10:50,696 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:10:50,704 [93] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:10:50,705 [93] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2022-05-19 06:10:50,705 [93] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2022-05-19 06:10:50,705 [93] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2022-05-19 06:10:50,705 [93] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:10:50,706 [93] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:10:50,706 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:10:50,706 [93] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 46, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 251, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2022-05-19 06:10:50,706 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:11:20 UTC)" executed successfully securityscanningnotificationworker stdout | 2022-05-19 06:10:51,479 [81] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-19 06:10:51,480 [81] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:07.583673+00:00 (in 16.103687 seconds) securityscanningnotificationworker stdout | 2022-05-19 06:10:51,480 [81] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:51 UTC)" (scheduled at 2022-05-19 06:10:51.479280+00:00) securityscanningnotificationworker stdout | 2022-05-19 06:10:51,480 [81] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-19 06:10:51,482 [81] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 10, 51, 480915), True, datetime.datetime(2022, 5, 19, 6, 10, 51, 480915), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-19 06:10:51,493 [81] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-19 06:10:51,493 [81] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-19 06:10:51,493 [81] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:51 UTC)" executed successfully builder stdout | 2022-05-19 06:10:51,577 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue chunkcleanupworker stdout | 2022-05-19 06:10:51,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:10:51,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:10:59.780399+00:00 (in 8.201367 seconds) chunkcleanupworker stdout | 2022-05-19 06:10:51,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:01 UTC)" (scheduled at 2022-05-19 06:10:51.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:10:51,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:10:51,581 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 10, 51, 579732), True, datetime.datetime(2022, 5, 19, 6, 10, 51, 579732), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:10:51,591 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:10:51,591 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:10:51,591 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:01 UTC)" executed successfully builder stdout | 2022-05-19 06:10:51,591 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:10:51,591 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:10:51,599 [56] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-19 06:10:52,310 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'gcworker.py', 'pid': '62'} namespacegcworker stdout | 2022-05-19 06:10:53,633 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'namespacegcworker.py', 'pid': '71'} servicekey stdout | 2022-05-19 06:10:53,724 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'servicekeyworker.py', 'pid': '94'} repositoryactioncounter stdout | 2022-05-19 06:10:57,941 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositoryactioncounter.py', 'pid': '79'} blobuploadcleanupworker stdout | 2022-05-19 06:10:57,950 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} exportactionlogsworker stdout | 2022-05-19 06:10:58,193 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-19 06:10:58,193 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:09.792901+00:00 (in 11.599393 seconds) exportactionlogsworker stdout | 2022-05-19 06:10:58,194 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-19 06:15:58 UTC)" (scheduled at 2022-05-19 06:10:58.192705+00:00) exportactionlogsworker stdout | 2022-05-19 06:10:58,195 [61] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 19, 6, 10, 58, 194279), 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-05-19 06:10:58,212 [61] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 19, 6, 10, 58, 194279), True, datetime.datetime(2022, 5, 19, 6, 10, 58, 194279), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-05-19 06:10:58,215 [61] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 19, 6, 10, 58, 194279), True, datetime.datetime(2022, 5, 19, 6, 10, 58, 194279), 0, 'exportactionlogs/%', False, datetime.datetime(2022, 5, 19, 6, 10, 58, 194279), 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-05-19 06:10:58,218 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-19 06:10:58,218 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-19 06:15:58 UTC)" executed successfully storagereplication stdout | 2022-05-19 06:10:58,425 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'storagereplication.py', 'pid': '97'} securityworker stdout | 2022-05-19 06:10:58,791 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityworker.py', 'pid': '93'} notificationworker stdout | 2022-05-19 06:10:59,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:10:59,299 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:09.297880+00:00 (in 9.999124 seconds) notificationworker stdout | 2022-05-19 06:10:59,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:09 UTC)" (scheduled at 2022-05-19 06:10:59.297880+00:00) notificationworker stdout | 2022-05-19 06:10:59,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:10:59,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 10, 59, 299482), True, datetime.datetime(2022, 5, 19, 6, 10, 59, 299482), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:10:59,312 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:10:59,312 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:10:59,312 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:09 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:10:59,780 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:10:59,781 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:01.578357+00:00 (in 1.797221 seconds) chunkcleanupworker stdout | 2022-05-19 06:10:59,781 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:59 UTC)" (scheduled at 2022-05-19 06:10:59.780399+00:00) chunkcleanupworker stdout | 2022-05-19 06:10:59,781 [58] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2022-05-19 06:10:59,781 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:11:59 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:10:59,913 [235] [DEBUG] [app] Starting request: urn:request:9677c36a-08f9-450f-8c5c-bcf4997467d9 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:10:59,913 [231] [DEBUG] [app] Starting request: urn:request:dc004dee-b185-456d-8173-042a0abc7881 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:10:59,916 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:10:59,916 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:10:59,919 [234] [DEBUG] [app] Starting request: urn:request:5ed2c5e1-3f9a-4df6-b0d1-30f1702a7192 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:10:59,919 [241] [DEBUG] [app] Starting request: urn:request:a53d6e8c-92d3-411c-801f-98a3783d0487 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:10:59,920 [241] [DEBUG] [app] Ending request: urn:request:a53d6e8c-92d3-411c-801f-98a3783d0487 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a53d6e8c-92d3-411c-801f-98a3783d0487', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:10:59,920 [234] [DEBUG] [app] Ending request: urn:request:5ed2c5e1-3f9a-4df6-b0d1-30f1702a7192 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5ed2c5e1-3f9a-4df6-b0d1-30f1702a7192', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:10:59,920 [234] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:10:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:10:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.002) nginx stdout | 127.0.0.1 () - - [19/May/2022:06:10:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.002) gunicorn-web stdout | 2022-05-19 06:10:59,921 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:10:59,921 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-19 06:10:59,920 [241] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:10:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:10:59,923 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:10:59,924 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:10:59,926 [231] [DEBUG] [app] Starting request: urn:request:a09f3920-bf8a-4181-8c4d-4724395eec4d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:10:59,927 [231] [DEBUG] [app] Ending request: urn:request:a09f3920-bf8a-4181-8c4d-4724395eec4d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a09f3920-bf8a-4181-8c4d-4724395eec4d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:10:59,927 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:10:59 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:10:59 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2022-05-19 06:10:59,928 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:10:59,929 [235] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:10:59,929 [231] [DEBUG] [app] Starting request: urn:request:28855a39-356c-43f0-8b8f-d656a56629d7 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:10:59,929 [231] [DEBUG] [app] Ending request: urn:request:28855a39-356c-43f0-8b8f-d656a56629d7 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:28855a39-356c-43f0-8b8f-d656a56629d7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:10:59,930 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:10:59 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:10:59 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2022-05-19 06:10:59,930 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:10:59,931 [231] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:10:59,935 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:10:59,937 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:10:59,945 [235] [DEBUG] [app] Ending request: urn:request:9677c36a-08f9-450f-8c5c-bcf4997467d9 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9677c36a-08f9-450f-8c5c-bcf4997467d9', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:10:59,945 [231] [DEBUG] [app] Ending request: urn:request:dc004dee-b185-456d-8173-042a0abc7881 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:dc004dee-b185-456d-8173-042a0abc7881', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:10:59,945 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:10:59,945 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:10:59,946 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:10:59 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:10:59 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.035 118 0.034) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:10:59 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.035 118 0.034) gunicorn-web stdout | 2022-05-19 06:10:59,946 [231] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:10:59 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" buildlogsarchiver stdout | 2022-05-19 06:11:00,548 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} queuecleanupworker stdout | 2022-05-19 06:11:00,738 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'queuecleanupworker.py', 'pid': '75'} globalpromstats stdout | 2022-05-19 06:11:00,934 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'globalpromstats.py', 'pid': '63'} teamsyncworker stdout | 2022-05-19 06:11:01,237 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'teamsyncworker.py', 'pid': '101'} chunkcleanupworker stdout | 2022-05-19 06:11:01,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:11:01,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:03.778152+00:00 (in 2.199083 seconds) chunkcleanupworker stdout | 2022-05-19 06:11:01,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:11 UTC)" (scheduled at 2022-05-19 06:11:01.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:11:01,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:11:01,580 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 11, 1, 579903), True, datetime.datetime(2022, 5, 19, 6, 11, 1, 579903), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:11:01,591 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:11:01,592 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:11:01,592 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:11 UTC)" executed successfully builder stdout | 2022-05-19 06:11:01,608 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:11:01,621 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:11:01,622 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:11:01,630 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:11:02,751 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} expiredappspecifictokenworker stdout | 2022-05-19 06:11:03,059 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} notificationworker stdout | 2022-05-19 06:11:03,214 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'notificationworker.py', 'pid': '73'} builder stdout | 2022-05-19 06:11:03,250 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'builder.py', 'pid': '56'} repositorygcworker stdout | 2022-05-19 06:11:03,510 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositorygcworker.py', 'pid': '80'} chunkcleanupworker stdout | 2022-05-19 06:11:03,778 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:11:03,779 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:11.578357+00:00 (in 7.799221 seconds) chunkcleanupworker stdout | 2022-05-19 06:11:03,779 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-19 06:16:03 UTC)" (scheduled at 2022-05-19 06:11:03.778152+00:00) chunkcleanupworker stdout | 2022-05-19 06:11:03,780 [58] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 19, 6, 11, 3, 779832), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2022-05-19 06:11:03,793 [58] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 19, 6, 11, 3, 779832), True, datetime.datetime(2022, 5, 19, 6, 11, 3, 779832), 0, 'chunk_cleanup/%']) chunkcleanupworker stdout | 2022-05-19 06:11:03,796 [58] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 19, 6, 11, 3, 779832), True, datetime.datetime(2022, 5, 19, 6, 11, 3, 779832), 0, 'chunk_cleanup/%', False, datetime.datetime(2022, 5, 19, 6, 11, 3, 779832), 'chunk_cleanup/%']) chunkcleanupworker stdout | 2022-05-19 06:11:03,799 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:11:03,799 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-19 06:16:03 UTC)" executed successfully manifestbackfillworker stdout | 2022-05-19 06:11:05,922 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} securityscanningnotificationworker stdout | 2022-05-19 06:11:07,034 [81] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityscanningnotificationworker.py', 'pid': '81'} logrotateworker stdout | 2022-05-19 06:11:07,462 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'logrotateworker.py', 'pid': '68'} securityscanningnotificationworker stdout | 2022-05-19 06:11:07,584 [81] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-19 06:11:07,584 [81] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:51.479280+00:00 (in 43.894703 seconds) securityscanningnotificationworker stdout | 2022-05-19 06:11:07,585 [81] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:07 UTC)" (scheduled at 2022-05-19 06:11:07.583673+00:00) securityscanningnotificationworker stdout | 2022-05-19 06:11:07,585 [81] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-19 06:11:07,585 [81] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:07 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:11:08,042 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '231'} gunicorn-web stdout | 2022-05-19 06:11:08,049 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '230'} gunicorn-web stdout | 2022-05-19 06:11:08,061 [235] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '235'} gunicorn-web stdout | 2022-05-19 06:11:08,069 [233] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '233'} gunicorn-web stdout | 2022-05-19 06:11:08,081 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-19 06:11:09,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:11:09,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:19.297880+00:00 (in 9.999236 seconds) notificationworker stdout | 2022-05-19 06:11:09,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:19 UTC)" (scheduled at 2022-05-19 06:11:09.297880+00:00) notificationworker stdout | 2022-05-19 06:11:09,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:11:09,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 11, 9, 299236), True, datetime.datetime(2022, 5, 19, 6, 11, 9, 299236), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:11:09,319 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:11:09,319 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:11:09,319 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:19 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-19 06:11:09,793 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-19 06:11:09,794 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:36.193076+00:00 (in 26.399268 seconds) exportactionlogsworker stdout | 2022-05-19 06:11:09,794 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:09 UTC)" (scheduled at 2022-05-19 06:11:09.792901+00:00) exportactionlogsworker stdout | 2022-05-19 06:11:09,794 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-19 06:11:09,795 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 11, 9, 794459), True, datetime.datetime(2022, 5, 19, 6, 11, 9, 794459), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-19 06:11:09,806 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-19 06:11:09,806 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-19 06:11:09,807 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:09 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:11:11,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:11:11,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:21.578357+00:00 (in 9.999156 seconds) chunkcleanupworker stdout | 2022-05-19 06:11:11,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:21 UTC)" (scheduled at 2022-05-19 06:11:11.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:11:11,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:11:11,581 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 11, 11, 579942), True, datetime.datetime(2022, 5, 19, 6, 11, 11, 579942), 0, 'chunk_cleanup/%', 50, 1, 0]) gcworker stdout | 2022-05-19 06:11:11,585 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-19 06:11:11,585 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:41.584428+00:00 (in 29.999052 seconds) gcworker stdout | 2022-05-19 06:11:11,585 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:11:41 UTC)" (scheduled at 2022-05-19 06:11:11.584428+00:00) gcworker stdout | 2022-05-19 06:11:11,586 [62] [DEBUG] [__main__] No GC policies found gcworker stdout | 2022-05-19 06:11:11,586 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:11:41 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:11:11,593 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:11:11,594 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:11:11,594 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:21 UTC)" executed successfully builder stdout | 2022-05-19 06:11:11,641 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:11:11,654 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:11:11,654 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:11:11,663 [56] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-19 06:11:12,593 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-19 06:11:12,599 [234] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '234'} gunicorn-registry stdout | 2022-05-19 06:11:12,604 [238] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '238'} gunicorn-registry stdout | 2022-05-19 06:11:12,611 [232] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '232'} gunicorn-registry stdout | 2022-05-19 06:11:12,620 [241] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '241'} gunicorn-registry stdout | 2022-05-19 06:11:12,627 [240] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '240'} gunicorn-registry stdout | 2022-05-19 06:11:12,633 [236] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '236'} gunicorn-registry stdout | 2022-05-19 06:11:12,639 [239] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '239'} gunicorn-registry stdout | 2022-05-19 06:11:12,644 [237] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '237'} securityworker stdout | 2022-05-19 06:11:13,479 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:11:13,480 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:20.688187+00:00 (in 7.208399 seconds) securityworker stdout | 2022-05-19 06:11:13,480 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:11:43 UTC)" (scheduled at 2022-05-19 06:11:13.478958+00:00) securityworker stdout | 2022-05-19 06:11:13,480 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:11:13,480 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:11:13,483 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:11:13,486 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:11:13,496 [93] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:11:13,498 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:11:13,499 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:11:43 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:11:14,912 [231] [DEBUG] [app] Starting request: urn:request:609020eb-4e99-46db-8ffc-3f78fe08c76e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:11:14,912 [233] [DEBUG] [app] Starting request: urn:request:2fb4a5e2-f04b-4a4c-afa6-6b49c6672360 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:11:14,915 [233] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:11:14,915 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:11:14,918 [232] [DEBUG] [app] Starting request: urn:request:eca42eb4-743c-4cb4-a779-91163e3bd57b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:11:14,919 [232] [DEBUG] [app] Ending request: urn:request:eca42eb4-743c-4cb4-a779-91163e3bd57b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:eca42eb4-743c-4cb4-a779-91163e3bd57b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:11:14,919 [232] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:11:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:11:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2022-05-19 06:11:14,919 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-19 06:11:14,920 [236] [DEBUG] [app] Starting request: urn:request:5fab5d37-7a90-4858-93a7-b4e1e8150a9b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:11:14,922 [236] [DEBUG] [app] Ending request: urn:request:5fab5d37-7a90-4858-93a7-b4e1e8150a9b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5fab5d37-7a90-4858-93a7-b4e1e8150a9b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:11:14,922 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost nginx stdout | 127.0.0.1 () - - [19/May/2022:06:11:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.006 162 0.006) gunicorn-registry stdout | 2022-05-19 06:11:14,923 [236] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:11:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:11:14,923 [233] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:11:14,925 [231] [DEBUG] [app] Starting request: urn:request:21e0b1a8-11e4-4bdf-8060-db09f93a1e62 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:11:14,925 [231] [DEBUG] [app] Ending request: urn:request:21e0b1a8-11e4-4bdf-8060-db09f93a1e62 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:21e0b1a8-11e4-4bdf-8060-db09f93a1e62', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:11:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.002) gunicorn-web stdout | 2022-05-19 06:11:14,926 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:11:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:11:14,926 [233] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:11:14,926 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:11:14,927 [231] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:11:14,928 [235] [DEBUG] [app] Starting request: urn:request:ca66e856-6c8b-4ebd-8ef3-0aff145996d3 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:11:14,929 [235] [DEBUG] [app] Ending request: urn:request:ca66e856-6c8b-4ebd-8ef3-0aff145996d3 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ca66e856-6c8b-4ebd-8ef3-0aff145996d3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:11:14,929 [235] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:11:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:11:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2022-05-19 06:11:14,930 [233] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:11:14,931 [233] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:11:14,934 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:11:14,938 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:11:14,943 [231] [DEBUG] [app] Ending request: urn:request:609020eb-4e99-46db-8ffc-3f78fe08c76e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:609020eb-4e99-46db-8ffc-3f78fe08c76e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:11:14,943 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:11:14,944 [231] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:11:14 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:11:14 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.033 118 0.032) gunicorn-web stdout | 2022-05-19 06:11:14,948 [233] [DEBUG] [app] Ending request: urn:request:2fb4a5e2-f04b-4a4c-afa6-6b49c6672360 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:2fb4a5e2-f04b-4a4c-afa6-6b49c6672360', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:11:14,948 [233] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:11:14,949 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:11:14 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:11:14 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.038 118 0.037) buildlogsarchiver stdout | 2022-05-19 06:11:15,474 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-19 06:11:15,474 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:45.473591+00:00 (in 29.999104 seconds) buildlogsarchiver stdout | 2022-05-19 06:11:15,475 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:11:45 UTC)" (scheduled at 2022-05-19 06:11:15.473591+00:00) buildlogsarchiver stdout | 2022-05-19 06:11:15,476 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 5, 4, 6, 11, 15, 475228), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-19 06:11:15,488 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-19 06:11:15,488 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-19 06:11:15,489 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:11:45 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-19 06:11:16,359 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} namespacegcworker stdout | 2022-05-19 06:11:16,680 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-19 06:11:16,680 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:46.491684+00:00 (in 29.811342 seconds) namespacegcworker stdout | 2022-05-19 06:11:16,680 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:16 UTC)" (scheduled at 2022-05-19 06:11:16.679426+00:00) namespacegcworker stdout | 2022-05-19 06:11:16,681 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-19 06:11:16,681 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:16 UTC)" executed successfully notificationworker stdout | 2022-05-19 06:11:19,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:11:19,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:29.297880+00:00 (in 9.999275 seconds) notificationworker stdout | 2022-05-19 06:11:19,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:29 UTC)" (scheduled at 2022-05-19 06:11:19.297880+00:00) notificationworker stdout | 2022-05-19 06:11:19,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:11:19,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 11, 19, 299333), True, datetime.datetime(2022, 5, 19, 6, 11, 19, 299333), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:11:19,318 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:11:19,318 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:11:19,318 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:29 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-19 06:11:20,185 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-19 06:11:20,194 [187] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '187'} gunicorn-secscan stdout | 2022-05-19 06:11:20,201 [188] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '188'} securityworker stdout | 2022-05-19 06:11:20,688 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:11:20,689 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:43.478958+00:00 (in 22.790091 seconds) securityworker stdout | 2022-05-19 06:11:20,689 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:11:50 UTC)" (scheduled at 2022-05-19 06:11:20.688187+00:00) securityworker stdout | 2022-05-19 06:11:20,689 [93] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:11:20,689 [93] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2022-05-19 06:11:20,690 [93] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:11:20,690 [93] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2022-05-19 06:11:20,691 [93] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:11:20,691 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:11:20,691 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:11:20,694 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:11:20,695 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:11:20,704 [93] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:11:20,705 [93] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2022-05-19 06:11:20,705 [93] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2022-05-19 06:11:20,705 [93] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2022-05-19 06:11:20,705 [93] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:11:20,706 [93] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:11:20,706 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:11:20,706 [93] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 46, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 251, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2022-05-19 06:11:20,706 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:11:50 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:11:21,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:11:21,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:31.578357+00:00 (in 9.999300 seconds) chunkcleanupworker stdout | 2022-05-19 06:11:21,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:31 UTC)" (scheduled at 2022-05-19 06:11:21.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:11:21,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:11:21,580 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 11, 21, 579647), True, datetime.datetime(2022, 5, 19, 6, 11, 21, 579647), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:11:21,591 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:11:21,591 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:11:21,592 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:31 UTC)" executed successfully builder stdout | 2022-05-19 06:11:21,674 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:11:21,687 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:11:21,688 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:11:21,698 [56] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-19 06:11:22,354 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'gcworker.py', 'pid': '62'} namespacegcworker stdout | 2022-05-19 06:11:23,661 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'namespacegcworker.py', 'pid': '71'} servicekey stdout | 2022-05-19 06:11:23,757 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'servicekeyworker.py', 'pid': '94'} repositoryactioncounter stdout | 2022-05-19 06:11:27,980 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositoryactioncounter.py', 'pid': '79'} blobuploadcleanupworker stdout | 2022-05-19 06:11:27,989 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} storagereplication stdout | 2022-05-19 06:11:28,455 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'storagereplication.py', 'pid': '97'} securityworker stdout | 2022-05-19 06:11:28,825 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityworker.py', 'pid': '93'} notificationworker stdout | 2022-05-19 06:11:29,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:11:29,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:39.297880+00:00 (in 9.999300 seconds) notificationworker stdout | 2022-05-19 06:11:29,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:39 UTC)" (scheduled at 2022-05-19 06:11:29.297880+00:00) notificationworker stdout | 2022-05-19 06:11:29,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:11:29,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 11, 29, 299339), True, datetime.datetime(2022, 5, 19, 6, 11, 29, 299339), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:11:29,314 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:11:29,314 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:11:29,314 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:39 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:11:29,912 [230] [DEBUG] [app] Starting request: urn:request:389b988d-b18e-4276-95c8-748ef187ccc4 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:11:29,912 [235] [DEBUG] [app] Starting request: urn:request:65972746-fd82-4146-b1ae-bd83f235a881 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:11:29,914 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:11:29,914 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:11:29,916 [232] [DEBUG] [app] Starting request: urn:request:513fcb0e-bb28-47ba-81b8-eb054c38e748 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:11:29,917 [238] [DEBUG] [app] Starting request: urn:request:318934ad-5e62-4959-b351-2abf899a1caa (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:11:29,918 [238] [DEBUG] [app] Ending request: urn:request:318934ad-5e62-4959-b351-2abf899a1caa (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:318934ad-5e62-4959-b351-2abf899a1caa', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:11:29,918 [238] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:11:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:11:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.002) gunicorn-registry stdout | 2022-05-19 06:11:29,918 [232] [DEBUG] [app] Ending request: urn:request:513fcb0e-bb28-47ba-81b8-eb054c38e748 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:513fcb0e-bb28-47ba-81b8-eb054c38e748', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:11:29,918 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [19/May/2022:06:11:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.004 162 0.003) gunicorn-registry stdout | 2022-05-19 06:11:29,919 [232] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:11:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:11:29,919 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:11:29,921 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:11:29,921 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:11:29,923 [230] [DEBUG] [app] Starting request: urn:request:4ecd7a7f-2e75-4a85-bc38-5260e64266a2 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:11:29,923 [235] [DEBUG] [app] Starting request: urn:request:03ab697a-68f7-4808-a9f1-07c1399ebb2e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:11:29,924 [230] [DEBUG] [app] Ending request: urn:request:4ecd7a7f-2e75-4a85-bc38-5260e64266a2 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4ecd7a7f-2e75-4a85-bc38-5260e64266a2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:11:29,924 [235] [DEBUG] [app] Ending request: urn:request:03ab697a-68f7-4808-a9f1-07c1399ebb2e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:03ab697a-68f7-4808-a9f1-07c1399ebb2e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:11:29,924 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:11:29 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:11:29,924 [235] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:11:29 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:11:29 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [19/May/2022:06:11:29 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.001) gunicorn-web stdout | 2022-05-19 06:11:29,924 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:11:29,925 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:11:29,925 [230] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:11:29,925 [235] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:11:29,933 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:11:29,933 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:11:29,942 [235] [DEBUG] [app] Ending request: urn:request:65972746-fd82-4146-b1ae-bd83f235a881 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:65972746-fd82-4146-b1ae-bd83f235a881', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:11:29,942 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:11:29,942 [230] [DEBUG] [app] Ending request: urn:request:389b988d-b18e-4276-95c8-748ef187ccc4 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:389b988d-b18e-4276-95c8-748ef187ccc4', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:11:29,943 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:11:29 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-19 06:11:29,943 [230] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:11:29 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.031 118 0.032) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:11:29 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.033 118 0.032) gunicorn-web stdout | 2022-05-19 06:11:29,943 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:11:29 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" repositorygcworker stdout | 2022-05-19 06:11:29,976 [80] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-19 06:11:29,977 [80] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:31.173528+00:00 (in 1.196614 seconds) repositorygcworker stdout | 2022-05-19 06:11:29,977 [80] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:29 UTC)" (scheduled at 2022-05-19 06:11:29.976033+00:00) repositorygcworker stdout | 2022-05-19 06:11:29,977 [80] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-19 06:11:29,979 [80] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 11, 29, 977805), True, datetime.datetime(2022, 5, 19, 6, 11, 29, 977805), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-19 06:11:29,991 [80] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-19 06:11:29,991 [80] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-19 06:11:29,991 [80] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:29 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-19 06:11:30,590 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} queuecleanupworker stdout | 2022-05-19 06:11:30,772 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'queuecleanupworker.py', 'pid': '75'} globalpromstats stdout | 2022-05-19 06:11:30,945 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'globalpromstats.py', 'pid': '63'} repositorygcworker stdout | 2022-05-19 06:11:31,174 [80] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-19 06:11:31,174 [80] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:29.976033+00:00 (in 58.801774 seconds) repositorygcworker stdout | 2022-05-19 06:11:31,174 [80] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:31 UTC)" (scheduled at 2022-05-19 06:11:31.173528+00:00) repositorygcworker stdout | 2022-05-19 06:11:31,175 [80] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-19 06:11:31,175 [80] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:31 UTC)" executed successfully teamsyncworker stdout | 2022-05-19 06:11:31,278 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'teamsyncworker.py', 'pid': '101'} chunkcleanupworker stdout | 2022-05-19 06:11:31,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:11:31,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:41.578357+00:00 (in 9.999318 seconds) chunkcleanupworker stdout | 2022-05-19 06:11:31,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:41 UTC)" (scheduled at 2022-05-19 06:11:31.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:11:31,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:11:31,580 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 11, 31, 579950), True, datetime.datetime(2022, 5, 19, 6, 11, 31, 579950), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:11:31,594 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:11:31,594 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:11:31,595 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:41 UTC)" executed successfully builder stdout | 2022-05-19 06:11:31,699 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:11:31,712 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:11:31,712 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:11:31,721 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:11:32,780 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} expiredappspecifictokenworker stdout | 2022-05-19 06:11:33,090 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} notificationworker stdout | 2022-05-19 06:11:33,258 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'notificationworker.py', 'pid': '73'} builder stdout | 2022-05-19 06:11:33,266 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'builder.py', 'pid': '56'} repositorygcworker stdout | 2022-05-19 06:11:33,543 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositorygcworker.py', 'pid': '80'} manifestbackfillworker stdout | 2022-05-19 06:11:35,943 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} exportactionlogsworker stdout | 2022-05-19 06:11:36,193 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-19 06:11:36,194 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:09.792901+00:00 (in 33.598997 seconds) exportactionlogsworker stdout | 2022-05-19 06:11:36,194 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:36 UTC)" (scheduled at 2022-05-19 06:11:36.193076+00:00) exportactionlogsworker stdout | 2022-05-19 06:11:36,194 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-19 06:11:36,194 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:36 UTC)" executed successfully securityscanningnotificationworker stdout | 2022-05-19 06:11:37,052 [81] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityscanningnotificationworker.py', 'pid': '81'} logrotateworker stdout | 2022-05-19 06:11:37,474 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'logrotateworker.py', 'pid': '68'} gunicorn-web stdout | 2022-05-19 06:11:38,056 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '231'} gunicorn-web stdout | 2022-05-19 06:11:38,065 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '230'} gunicorn-web stdout | 2022-05-19 06:11:38,072 [235] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '235'} gunicorn-web stdout | 2022-05-19 06:11:38,083 [233] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '233'} gunicorn-web stdout | 2022-05-19 06:11:38,095 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-19 06:11:39,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:11:39,299 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:48.405016+00:00 (in 9.106176 seconds) notificationworker stdout | 2022-05-19 06:11:39,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:49 UTC)" (scheduled at 2022-05-19 06:11:39.297880+00:00) notificationworker stdout | 2022-05-19 06:11:39,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:11:39,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 11, 39, 299627), True, datetime.datetime(2022, 5, 19, 6, 11, 39, 299627), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:11:39,321 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:11:39,322 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:11:39,322 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:49 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:11:41,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:11:41,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:51.578357+00:00 (in 9.999205 seconds) chunkcleanupworker stdout | 2022-05-19 06:11:41,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:51 UTC)" (scheduled at 2022-05-19 06:11:41.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:11:41,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:11:41,581 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 11, 41, 579947), True, datetime.datetime(2022, 5, 19, 6, 11, 41, 579947), 0, 'chunk_cleanup/%', 50, 1, 0]) gcworker stdout | 2022-05-19 06:11:41,585 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-19 06:11:41,585 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:11.584428+00:00 (in 29.999148 seconds) gcworker stdout | 2022-05-19 06:11:41,585 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:12:11 UTC)" (scheduled at 2022-05-19 06:11:41.584428+00:00) gcworker stdout | 2022-05-19 06:11:41,586 [62] [DEBUG] [__main__] No GC policies found gcworker stdout | 2022-05-19 06:11:41,586 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:12:11 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:11:41,593 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:11:41,594 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:11:41,594 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:51 UTC)" executed successfully builder stdout | 2022-05-19 06:11:41,732 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:11:41,746 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:11:41,746 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:11:41,757 [56] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-19 06:11:42,608 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-19 06:11:42,612 [234] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '234'} gunicorn-registry stdout | 2022-05-19 06:11:42,620 [238] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '238'} gunicorn-registry stdout | 2022-05-19 06:11:42,625 [232] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '232'} gunicorn-registry stdout | 2022-05-19 06:11:42,633 [241] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '241'} gunicorn-registry stdout | 2022-05-19 06:11:42,641 [240] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '240'} gunicorn-registry stdout | 2022-05-19 06:11:42,647 [236] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '236'} gunicorn-registry stdout | 2022-05-19 06:11:42,655 [239] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '239'} gunicorn-registry stdout | 2022-05-19 06:11:42,660 [237] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '237'} securityworker stdout | 2022-05-19 06:11:43,479 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:11:43,480 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:50.688187+00:00 (in 7.208431 seconds) securityworker stdout | 2022-05-19 06:11:43,480 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:12:13 UTC)" (scheduled at 2022-05-19 06:11:43.478958+00:00) securityworker stdout | 2022-05-19 06:11:43,480 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:11:43,480 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:11:43,483 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:11:43,485 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:11:43,495 [93] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:11:43,497 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:11:43,498 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:12:13 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:11:44,913 [235] [DEBUG] [app] Starting request: urn:request:6e56d991-58dc-460a-9a17-e6eda74640bf (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:11:44,913 [230] [DEBUG] [app] Starting request: urn:request:7a10d606-d7cd-46d6-88e4-702ac7d16fd6 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:11:44,915 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:11:44,915 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:11:44,918 [238] [DEBUG] [app] Starting request: urn:request:f2bffb9c-7634-4d9e-881d-59af12b4323d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:11:44,918 [241] [DEBUG] [app] Starting request: urn:request:5f1b6f18-2be8-4688-9f75-68831622c0e7 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:11:44,919 [241] [DEBUG] [app] Ending request: urn:request:5f1b6f18-2be8-4688-9f75-68831622c0e7 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5f1b6f18-2be8-4688-9f75-68831622c0e7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:11:44,919 [238] [DEBUG] [app] Ending request: urn:request:f2bffb9c-7634-4d9e-881d-59af12b4323d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f2bffb9c-7634-4d9e-881d-59af12b4323d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:11:44,920 [238] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:11:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2022-05-19 06:11:44,920 [241] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:11:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:11:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.004 162 0.003) nginx stdout | 127.0.0.1 () - - [19/May/2022:06:11:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.003) gunicorn-web stdout | 2022-05-19 06:11:44,921 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:11:44,921 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:11:44,923 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:11:44,923 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:11:44,925 [230] [DEBUG] [app] Starting request: urn:request:b7f53612-29fa-44a5-a119-d12d21125a74 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:11:44,926 [231] [DEBUG] [app] Starting request: urn:request:a0f3cf43-08d9-4d71-8c35-069a7300240e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:11:44,926 [230] [DEBUG] [app] Ending request: urn:request:b7f53612-29fa-44a5-a119-d12d21125a74 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b7f53612-29fa-44a5-a119-d12d21125a74', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:11:44,926 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:11:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:11:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2022-05-19 06:11:44,926 [231] [DEBUG] [app] Ending request: urn:request:a0f3cf43-08d9-4d71-8c35-069a7300240e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a0f3cf43-08d9-4d71-8c35-069a7300240e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:11:44,927 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:11:44,927 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:11:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:11:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.002) gunicorn-web stdout | 2022-05-19 06:11:44,927 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:11:44,927 [230] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:11:44,928 [235] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:11:44,935 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:11:44,935 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:11:44,944 [235] [DEBUG] [app] Ending request: urn:request:6e56d991-58dc-460a-9a17-e6eda74640bf (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6e56d991-58dc-460a-9a17-e6eda74640bf', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:11:44,944 [230] [DEBUG] [app] Ending request: urn:request:7a10d606-d7cd-46d6-88e4-702ac7d16fd6 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7a10d606-d7cd-46d6-88e4-702ac7d16fd6', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:11:44,944 [230] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:11:44,944 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:11:44,945 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:11:44 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-19 06:11:44,945 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:11:44 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:11:44 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.034 118 0.033) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:11:44 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.034 118 0.033) buildlogsarchiver stdout | 2022-05-19 06:11:45,474 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-19 06:11:45,475 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:15.473591+00:00 (in 29.998838 seconds) buildlogsarchiver stdout | 2022-05-19 06:11:45,475 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:12:15 UTC)" (scheduled at 2022-05-19 06:11:45.473591+00:00) buildlogsarchiver stdout | 2022-05-19 06:11:45,476 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 5, 4, 6, 11, 45, 475387), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-19 06:11:45,486 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-19 06:11:45,486 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-19 06:11:45,487 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:12:15 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-19 06:11:46,381 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} namespacegcworker stdout | 2022-05-19 06:11:46,492 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-19 06:11:46,492 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:16.679426+00:00 (in 30.186750 seconds) namespacegcworker stdout | 2022-05-19 06:11:46,493 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:46 UTC)" (scheduled at 2022-05-19 06:11:46.491684+00:00) namespacegcworker stdout | 2022-05-19 06:11:46,493 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-19 06:11:46,494 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 11, 46, 493417), True, datetime.datetime(2022, 5, 19, 6, 11, 46, 493417), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-19 06:11:46,505 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-19 06:11:46,505 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-19 06:11:46,505 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:46 UTC)" executed successfully notificationworker stdout | 2022-05-19 06:11:48,405 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:11:48,406 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:49.297880+00:00 (in 0.891988 seconds) notificationworker stdout | 2022-05-19 06:11:48,406 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:48 UTC)" (scheduled at 2022-05-19 06:11:48.405016+00:00) notificationworker stdout | 2022-05-19 06:11:48,407 [73] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-19 06:11:48,407 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:48 UTC)" executed successfully notificationworker stdout | 2022-05-19 06:11:49,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:11:49,299 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:59.297880+00:00 (in 9.998999 seconds) notificationworker stdout | 2022-05-19 06:11:49,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:59 UTC)" (scheduled at 2022-05-19 06:11:49.297880+00:00) notificationworker stdout | 2022-05-19 06:11:49,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:11:49,301 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 11, 49, 299888), True, datetime.datetime(2022, 5, 19, 6, 11, 49, 299888), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:11:49,312 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:11:49,312 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:11:49,312 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:11:59 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-19 06:11:50,199 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-19 06:11:50,208 [187] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '187'} gunicorn-secscan stdout | 2022-05-19 06:11:50,215 [188] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '188'} securityworker stdout | 2022-05-19 06:11:50,688 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:11:50,689 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:13.478958+00:00 (in 22.789904 seconds) securityworker stdout | 2022-05-19 06:11:50,689 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:12:20 UTC)" (scheduled at 2022-05-19 06:11:50.688187+00:00) securityworker stdout | 2022-05-19 06:11:50,689 [93] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:11:50,689 [93] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2022-05-19 06:11:50,691 [93] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:11:50,691 [93] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2022-05-19 06:11:50,691 [93] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:11:50,692 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:11:50,692 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:11:50,694 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:11:50,696 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:11:50,705 [93] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:11:50,706 [93] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2022-05-19 06:11:50,706 [93] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2022-05-19 06:11:50,706 [93] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2022-05-19 06:11:50,706 [93] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:11:50,707 [93] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:11:50,707 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:11:50,707 [93] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 46, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 251, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2022-05-19 06:11:50,707 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:12:20 UTC)" executed successfully securityscanningnotificationworker stdout | 2022-05-19 06:11:51,479 [81] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-19 06:11:51,480 [81] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:07.583673+00:00 (in 16.103534 seconds) securityscanningnotificationworker stdout | 2022-05-19 06:11:51,480 [81] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:51 UTC)" (scheduled at 2022-05-19 06:11:51.479280+00:00) securityscanningnotificationworker stdout | 2022-05-19 06:11:51,480 [81] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-19 06:11:51,482 [81] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 11, 51, 480856), True, datetime.datetime(2022, 5, 19, 6, 11, 51, 480856), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-19 06:11:51,492 [81] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-19 06:11:51,493 [81] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-19 06:11:51,493 [81] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:51 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:11:51,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:11:51,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:11:59.780399+00:00 (in 8.201378 seconds) chunkcleanupworker stdout | 2022-05-19 06:11:51,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:01 UTC)" (scheduled at 2022-05-19 06:11:51.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:11:51,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:11:51,580 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 11, 51, 579673), True, datetime.datetime(2022, 5, 19, 6, 11, 51, 579673), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:11:51,591 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:11:51,591 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:11:51,591 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:01 UTC)" executed successfully builder stdout | 2022-05-19 06:11:51,760 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:11:51,773 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:11:51,773 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:11:51,789 [56] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-19 06:11:52,387 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'gcworker.py', 'pid': '62'} namespacegcworker stdout | 2022-05-19 06:11:53,702 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'namespacegcworker.py', 'pid': '71'} servicekey stdout | 2022-05-19 06:11:53,798 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'servicekeyworker.py', 'pid': '94'} repositoryactioncounter stdout | 2022-05-19 06:11:58,017 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositoryactioncounter.py', 'pid': '79'} blobuploadcleanupworker stdout | 2022-05-19 06:11:58,023 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} storagereplication stdout | 2022-05-19 06:11:58,495 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'storagereplication.py', 'pid': '97'} securityworker stdout | 2022-05-19 06:11:58,866 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityworker.py', 'pid': '93'} notificationworker stdout | 2022-05-19 06:11:59,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:11:59,299 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:09.297880+00:00 (in 9.999101 seconds) notificationworker stdout | 2022-05-19 06:11:59,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:09 UTC)" (scheduled at 2022-05-19 06:11:59.297880+00:00) notificationworker stdout | 2022-05-19 06:11:59,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:11:59,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 11, 59, 299457), True, datetime.datetime(2022, 5, 19, 6, 11, 59, 299457), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:11:59,317 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:11:59,317 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:11:59,317 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:09 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:11:59,780 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:11:59,781 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:01.578357+00:00 (in 1.797350 seconds) chunkcleanupworker stdout | 2022-05-19 06:11:59,781 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:59 UTC)" (scheduled at 2022-05-19 06:11:59.780399+00:00) chunkcleanupworker stdout | 2022-05-19 06:11:59,781 [58] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2022-05-19 06:11:59,781 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:12:59 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:11:59,912 [235] [DEBUG] [app] Starting request: urn:request:02f4c3cf-9ea4-4a78-9b91-95f91abb96f3 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:11:59,913 [233] [DEBUG] [app] Starting request: urn:request:3028ef30-54d1-4587-b745-9b0f9ad29e5b (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:11:59,914 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:11:59,915 [233] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:11:59,917 [240] [DEBUG] [app] Starting request: urn:request:f212a026-eed6-4fc6-9b5d-b29d74a68ced (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:11:59,917 [241] [DEBUG] [app] Starting request: urn:request:f6cc334a-94b1-41d4-8444-b0690063bcae (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:11:59,918 [241] [DEBUG] [app] Ending request: urn:request:f6cc334a-94b1-41d4-8444-b0690063bcae (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f6cc334a-94b1-41d4-8444-b0690063bcae', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:11:59,918 [240] [DEBUG] [app] Ending request: urn:request:f212a026-eed6-4fc6-9b5d-b29d74a68ced (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f212a026-eed6-4fc6-9b5d-b29d74a68ced', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:11:59,918 [241] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:11:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2022-05-19 06:11:59,918 [240] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:11:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:11:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) nginx stdout | 127.0.0.1 () - - [19/May/2022:06:11:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-web stdout | 2022-05-19 06:11:59,919 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:11:59,919 [233] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:11:59,921 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:11:59,922 [233] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:11:59,923 [235] [DEBUG] [app] Starting request: urn:request:041be72b-c4ae-4b4e-b848-4db7696f2d6c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:11:59,924 [235] [DEBUG] [app] Ending request: urn:request:041be72b-c4ae-4b4e-b848-4db7696f2d6c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:041be72b-c4ae-4b4e-b848-4db7696f2d6c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:11:59,924 [233] [DEBUG] [app] Starting request: urn:request:3f2bf5e4-1b1a-4ac8-94f0-73c1d999f2c0 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:11:59,924 [235] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:11:59 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:11:59,925 [233] [DEBUG] [app] Ending request: urn:request:3f2bf5e4-1b1a-4ac8-94f0-73c1d999f2c0 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3f2bf5e4-1b1a-4ac8-94f0-73c1d999f2c0', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:11:59 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2022-05-19 06:11:59,925 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:11:59,925 [233] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:11:59 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:11:59 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2022-05-19 06:11:59,926 [233] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:11:59,926 [235] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:11:59,926 [233] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:11:59,933 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:11:59,933 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:11:59,941 [235] [DEBUG] [app] Ending request: urn:request:02f4c3cf-9ea4-4a78-9b91-95f91abb96f3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:02f4c3cf-9ea4-4a78-9b91-95f91abb96f3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:11:59,941 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:11:59,941 [233] [DEBUG] [app] Ending request: urn:request:3028ef30-54d1-4587-b745-9b0f9ad29e5b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3028ef30-54d1-4587-b745-9b0f9ad29e5b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:11:59,941 [233] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:11:59,941 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:11:59 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:11:59 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.030 118 0.031) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:11:59 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.030 118 0.030) gunicorn-web stdout | 2022-05-19 06:11:59,942 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:11:59 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" buildlogsarchiver stdout | 2022-05-19 06:12:00,625 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} queuecleanupworker stdout | 2022-05-19 06:12:00,809 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'queuecleanupworker.py', 'pid': '75'} globalpromstats stdout | 2022-05-19 06:12:00,962 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'globalpromstats.py', 'pid': '63'} teamsyncworker stdout | 2022-05-19 06:12:01,319 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'teamsyncworker.py', 'pid': '101'} chunkcleanupworker stdout | 2022-05-19 06:12:01,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:12:01,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:11.578357+00:00 (in 9.999402 seconds) chunkcleanupworker stdout | 2022-05-19 06:12:01,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:11 UTC)" (scheduled at 2022-05-19 06:12:01.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:12:01,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:12:01,580 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 12, 1, 579494), True, datetime.datetime(2022, 5, 19, 6, 12, 1, 579494), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:12:01,591 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:12:01,591 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:12:01,591 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:11 UTC)" executed successfully builder stdout | 2022-05-19 06:12:01,800 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:12:01,813 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:12:01,813 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:12:01,822 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:12:02,795 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} expiredappspecifictokenworker stdout | 2022-05-19 06:12:03,107 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} notificationworker stdout | 2022-05-19 06:12:03,292 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'notificationworker.py', 'pid': '73'} builder stdout | 2022-05-19 06:12:03,297 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'builder.py', 'pid': '56'} repositorygcworker stdout | 2022-05-19 06:12:03,570 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositorygcworker.py', 'pid': '80'} manifestbackfillworker stdout | 2022-05-19 06:12:05,970 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} securityscanningnotificationworker stdout | 2022-05-19 06:12:07,094 [81] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityscanningnotificationworker.py', 'pid': '81'} logrotateworker stdout | 2022-05-19 06:12:07,492 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'logrotateworker.py', 'pid': '68'} securityscanningnotificationworker stdout | 2022-05-19 06:12:07,584 [81] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-19 06:12:07,584 [81] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:51.479280+00:00 (in 43.894584 seconds) securityscanningnotificationworker stdout | 2022-05-19 06:12:07,585 [81] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:07 UTC)" (scheduled at 2022-05-19 06:12:07.583673+00:00) securityscanningnotificationworker stdout | 2022-05-19 06:12:07,585 [81] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-19 06:12:07,585 [81] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:07 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:12:08,069 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '231'} gunicorn-web stdout | 2022-05-19 06:12:08,079 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '230'} gunicorn-web stdout | 2022-05-19 06:12:08,085 [235] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '235'} gunicorn-web stdout | 2022-05-19 06:12:08,094 [233] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '233'} gunicorn-web stdout | 2022-05-19 06:12:08,113 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '66'} gunicorn-web stdout | 2022-05-19 06:12:08,753 [230] [DEBUG] [app] Starting request: urn:request:d5f6b4d2-5906-4905-8b24-2a60734b2174 (/api/v1/user/notifications) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:08,753 [235] [DEBUG] [app] Starting request: urn:request:b7b92c72-f5bd-4532-bea2-748d9e7f514b (/api/v1/user/apptoken) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:08,756 [230] [DEBUG] [app] Ending request: urn:request:d5f6b4d2-5906-4905-8b24-2a60734b2174 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:d5f6b4d2-5906-4905-8b24-2a60734b2174', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:12:08,756 [235] [DEBUG] [app] Ending request: urn:request:b7b92c72-f5bd-4532-bea2-748d9e7f514b (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:b7b92c72-f5bd-4532-bea2-748d9e7f514b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:12:08,756 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:08 +0000] "GET /api/v1/user/notifications HTTP/1.0" 401 274 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:08 +0000] "GET /api/v1/user/notifications HTTP/1.1" 401 274 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.005 1640 0.005) gunicorn-web stdout | 2022-05-19 06:12:08,757 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:08 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 401 274 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:08 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.1" 401 274 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.006 1649 0.005) notificationworker stdout | 2022-05-19 06:12:09,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:12:09,299 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:19.297880+00:00 (in 9.999106 seconds) notificationworker stdout | 2022-05-19 06:12:09,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:19 UTC)" (scheduled at 2022-05-19 06:12:09.297880+00:00) notificationworker stdout | 2022-05-19 06:12:09,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:12:09,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 12, 9, 299473), True, datetime.datetime(2022, 5, 19, 6, 12, 9, 299473), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:12:09,315 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:12:09,316 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:12:09,316 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:19 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-19 06:12:09,793 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-19 06:12:09,794 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:36.193076+00:00 (in 26.399200 seconds) exportactionlogsworker stdout | 2022-05-19 06:12:09,794 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:09 UTC)" (scheduled at 2022-05-19 06:12:09.792901+00:00) exportactionlogsworker stdout | 2022-05-19 06:12:09,794 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-19 06:12:09,795 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 12, 9, 794880), True, datetime.datetime(2022, 5, 19, 6, 12, 9, 794880), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-19 06:12:09,808 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-19 06:12:09,808 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-19 06:12:09,808 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:09 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:12:11,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:12:11,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:21.578357+00:00 (in 9.999251 seconds) chunkcleanupworker stdout | 2022-05-19 06:12:11,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:21 UTC)" (scheduled at 2022-05-19 06:12:11.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:12:11,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:12:11,580 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 12, 11, 579743), True, datetime.datetime(2022, 5, 19, 6, 12, 11, 579743), 0, 'chunk_cleanup/%', 50, 1, 0]) gcworker stdout | 2022-05-19 06:12:11,585 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-19 06:12:11,585 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:41.584428+00:00 (in 29.999139 seconds) gcworker stdout | 2022-05-19 06:12:11,585 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:12:41 UTC)" (scheduled at 2022-05-19 06:12:11.584428+00:00) gcworker stdout | 2022-05-19 06:12:11,586 [62] [DEBUG] [__main__] No GC policies found gcworker stdout | 2022-05-19 06:12:11,586 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:12:41 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:12:11,591 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:12:11,591 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:12:11,591 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:21 UTC)" executed successfully builder stdout | 2022-05-19 06:12:11,823 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:12:11,836 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:12:11,837 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:12:11,845 [56] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-19 06:12:12,624 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-19 06:12:12,631 [234] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '234'} gunicorn-registry stdout | 2022-05-19 06:12:12,638 [238] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '238'} gunicorn-registry stdout | 2022-05-19 06:12:12,645 [232] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '232'} gunicorn-registry stdout | 2022-05-19 06:12:12,653 [241] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '241'} gunicorn-registry stdout | 2022-05-19 06:12:12,660 [240] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '240'} gunicorn-registry stdout | 2022-05-19 06:12:12,667 [236] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '236'} gunicorn-registry stdout | 2022-05-19 06:12:12,674 [239] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '239'} gunicorn-registry stdout | 2022-05-19 06:12:12,680 [237] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '237'} securityworker stdout | 2022-05-19 06:12:13,479 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:12:13,480 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:20.688187+00:00 (in 7.208425 seconds) securityworker stdout | 2022-05-19 06:12:13,480 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:12:43 UTC)" (scheduled at 2022-05-19 06:12:13.478958+00:00) securityworker stdout | 2022-05-19 06:12:13,480 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:12:13,480 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:12:13,484 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:12:13,486 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:12:13,497 [93] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:12:13,501 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:12:13,502 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:12:43 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:12:14,912 [230] [DEBUG] [app] Starting request: urn:request:88ed171b-984a-480c-b758-39f3b655855a (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:14,912 [235] [DEBUG] [app] Starting request: urn:request:c428c242-3bbb-4de1-8dc7-17565b391272 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:14,915 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:12:14,915 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:12:14,917 [232] [DEBUG] [app] Starting request: urn:request:674a3da2-966c-4f20-b2f6-d46b0a974a4b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:12:14,917 [240] [DEBUG] [app] Starting request: urn:request:7aabb6cb-f444-4114-b8c7-8a8c3230910f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:12:14,918 [232] [DEBUG] [app] Ending request: urn:request:674a3da2-966c-4f20-b2f6-d46b0a974a4b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:674a3da2-966c-4f20-b2f6-d46b0a974a4b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:12:14,918 [240] [DEBUG] [app] Ending request: urn:request:7aabb6cb-f444-4114-b8c7-8a8c3230910f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:7aabb6cb-f444-4114-b8c7-8a8c3230910f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:12:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) nginx stdout | 127.0.0.1 () - - [19/May/2022:06:12:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.002) gunicorn-registry stdout | 2022-05-19 06:12:14,919 [240] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:12:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:12:14,919 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-19 06:12:14,919 [232] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:12:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:12:14,919 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:12:14,921 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:12:14,921 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:12:14,923 [230] [DEBUG] [app] Starting request: urn:request:cc89d86a-6d41-42b0-bd1a-f0c2eb3dedad (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:12:14,924 [230] [DEBUG] [app] Ending request: urn:request:cc89d86a-6d41-42b0-bd1a-f0c2eb3dedad (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:cc89d86a-6d41-42b0-bd1a-f0c2eb3dedad', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:12:14,924 [235] [DEBUG] [app] Starting request: urn:request:a695fbef-0a61-498d-8dba-f8df0ba18166 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:12:14,924 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:12:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:12:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2022-05-19 06:12:14,925 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:12:14,925 [235] [DEBUG] [app] Ending request: urn:request:a695fbef-0a61-498d-8dba-f8df0ba18166 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a695fbef-0a61-498d-8dba-f8df0ba18166', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:12:14,925 [235] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:12:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:12:14,925 [230] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [19/May/2022:06:12:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2022-05-19 06:12:14,926 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:12:14,927 [235] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:12:14,934 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:12:14,934 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:12:14,943 [235] [DEBUG] [app] Ending request: urn:request:c428c242-3bbb-4de1-8dc7-17565b391272 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:c428c242-3bbb-4de1-8dc7-17565b391272', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:12:14,944 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:12:14,944 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:14 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-19 06:12:14,945 [230] [DEBUG] [app] Ending request: urn:request:88ed171b-984a-480c-b758-39f3b655855a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:88ed171b-984a-480c-b758-39f3b655855a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:14 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.033 118 0.034) gunicorn-web stdout | 2022-05-19 06:12:14,945 [230] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:12:14,946 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:14 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:14 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.035 118 0.035) gunicorn-web stdout | 2022-05-19 06:12:15,168 [233] [DEBUG] [app] Starting request: urn:request:991fc8ff-1a4b-4a59-a568-c8c854b54116 (/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:15,175 [233] [DEBUG] [app] Ending request: urn:request:991fc8ff-1a4b-4a59-a568-c8c854b54116 (/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7) {'endpoint': 'web.repository', 'request_id': 'urn:request:991fc8ff-1a4b-4a59-a568-c8c854b54116', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7', 'path': '/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:12:15,176 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:15 +0000] "GET /repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7 HTTP/1.0" 200 84286 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:15 +0000] "GET /repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7 HTTP/1.1" 200 13184 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.010 1686 0.010) buildlogsarchiver stdout | 2022-05-19 06:12:15,474 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-19 06:12:15,474 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:45.473591+00:00 (in 29.999257 seconds) buildlogsarchiver stdout | 2022-05-19 06:12:15,474 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:12:45 UTC)" (scheduled at 2022-05-19 06:12:15.473591+00:00) buildlogsarchiver stdout | 2022-05-19 06:12:15,475 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 5, 4, 6, 12, 15, 474938), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-19 06:12:15,487 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-19 06:12:15,488 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-19 06:12:15,488 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:12:45 UTC)" executed successfully nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:15 +0000] "GET /static/ldn/font-awesome.css-a166e3c966c6.css HTTP/1.1" 200 8417 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1262 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:15 +0000] "GET /static/ldn/cssfamilySourceSansPro300400700-30cd793a4b9b.css HTTP/1.1" 200 235 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1277 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:15 +0000] "GET /static/ldn/bootstrap.min.css-13f668494557.css HTTP/1.1" 200 24440 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1263 -) exportactionlogsworker stdout | 2022-05-19 06:12:16,423 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:16 +0000] "GET /static/ldn/bootstrap-datetimepicker.min.css-4b15a4ede750.css HTTP/1.1" 200 1482 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1278 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:16 +0000] "GET /static/ldn/cal-heatmap.css-904c3b9e8a60.css HTTP/1.1" 200 799 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1261 -) namespacegcworker stdout | 2022-05-19 06:12:16,680 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-19 06:12:16,680 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:46.491684+00:00 (in 29.811476 seconds) namespacegcworker stdout | 2022-05-19 06:12:16,680 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:16 UTC)" (scheduled at 2022-05-19 06:12:16.679426+00:00) namespacegcworker stdout | 2022-05-19 06:12:16,680 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-19 06:12:16,680 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:16 UTC)" executed successfully nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:16 +0000] "GET /static/ldn/ng-tags-input.min.css-7782190f81cd.css HTTP/1.1" 200 1079 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1267 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:17 +0000] "GET /static/build/83-quay-frontend.chunk.js HTTP/1.1" 200 1717 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1242 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:17 +0000] "GET /static/build/135-quay-frontend.chunk.js HTTP/1.1" 200 974 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1243 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:18 +0000] "GET /static/build/18-quay-frontend.chunk.js HTTP/1.1" 200 997 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1242 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:18 +0000] "GET /static/build/112-quay-frontend.chunk.js HTTP/1.1" 200 767 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1243 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:18 +0000] "GET /static/build/181-quay-frontend.chunk.js HTTP/1.1" 200 951 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1243 -) gunicorn-web stdout | 2022-05-19 06:12:18,721 [230] [DEBUG] [app] Starting request: urn:request:b89dabec-c28c-4bd3-aa93-77ff9445d0f3 (/api/v1/user/) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:18,723 [230] [DEBUG] [app] Ending request: urn:request:b89dabec-c28c-4bd3-aa93-77ff9445d0f3 (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:b89dabec-c28c-4bd3-aa93-77ff9445d0f3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:12:18,724 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:18 +0000] "GET /api/v1/user/ HTTP/1.0" 401 301 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:18 +0000] "GET /api/v1/user/ HTTP/1.1" 401 301 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.004 1356 0.005) gunicorn-web stdout | 2022-05-19 06:12:18,726 [231] [DEBUG] [app] Starting request: urn:request:89a062a0-421f-4118-ab17-4f0f496f0139 (/api/v1/messages) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:18,726 [233] [DEBUG] [app] Starting request: urn:request:58a10af0-5901-4718-a07f-21daf8024b53 (/api/v1/repository/build_test/github) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:18,727 [233] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2022-05-19 06:12:18,728 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."content", "t1"."uuid", "t1"."severity", "t1"."media_type_id", "t2"."id", "t2"."name" FROM "messages" AS "t1" INNER JOIN "mediatype" AS "t2" ON ("t1"."media_type_id" = "t2"."id")', []) gunicorn-web stdout | 2022-05-19 06:12:18,729 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t3"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 'public', 3, 1, 0]) gunicorn-web stdout | 2022-05-19 06:12:18,738 [231] [DEBUG] [app] Ending request: urn:request:89a062a0-421f-4118-ab17-4f0f496f0139 (/api/v1/messages) {'endpoint': 'api.globalusermessages', 'request_id': 'urn:request:89a062a0-421f-4118-ab17-4f0f496f0139', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/messages', 'path': '/api/v1/messages', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:12:18,739 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:12:18,740 [231] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:18 +0000] "GET /api/v1/messages HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:18 +0000] "GET /api/v1/messages HTTP/1.1" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.015 1359 0.015) gunicorn-web stdout | 2022-05-19 06:12:18,744 [233] [DEBUG] [app] Ending request: urn:request:58a10af0-5901-4718-a07f-21daf8024b53 (/api/v1/repository/build_test/github) {'endpoint': 'api.repository', 'request_id': 'urn:request:58a10af0-5901-4718-a07f-21daf8024b53', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/repository/build_test/github?includeTags=false', 'path': '/api/v1/repository/build_test/github', 'parameters': {'includeTags': 'false'}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:12:18,745 [233] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:18 +0000] "GET /api/v1/repository/build_test/github?includeTags=false HTTP/1.1" 401 274 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.020 1397 0.021) gunicorn-web stdout | 2022-05-19 06:12:18,745 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:18 +0000] "GET /api/v1/repository/build_test/github?includeTags=false HTTP/1.0" 401 274 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" notificationworker stdout | 2022-05-19 06:12:19,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:12:19,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:29.297880+00:00 (in 9.999224 seconds) notificationworker stdout | 2022-05-19 06:12:19,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:29 UTC)" (scheduled at 2022-05-19 06:12:19.297880+00:00) notificationworker stdout | 2022-05-19 06:12:19,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:12:19,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 12, 19, 299261), True, datetime.datetime(2022, 5, 19, 6, 12, 19, 299261), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:12:19,318 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:12:19,318 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:12:19,319 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:29 UTC)" executed successfully nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:20 +0000] "GET /static/build/152-quay-frontend.chunk.js HTTP/1.1" 200 313 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1243 -) gunicorn-web stdout | 2022-05-19 06:12:20,068 [231] [DEBUG] [app] Starting request: urn:request:f9355f86-9865-435e-aff6-c8fc0a7e36b1 (/api/v1/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:20,069 [231] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2022-05-19 06:12:20,071 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t3"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 'public', 3, 1, 0]) gunicorn-web stdout | 2022-05-19 06:12:20,085 [231] [DEBUG] [app] Ending request: urn:request:f9355f86-9865-435e-aff6-c8fc0a7e36b1 (/api/v1/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7) {'endpoint': 'api.repositorybuildresource', 'request_id': 'urn:request:f9355f86-9865-435e-aff6-c8fc0a7e36b1', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7', 'path': '/api/v1/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:12:20,086 [231] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:20 +0000] "GET /api/v1/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7 HTTP/1.1" 401 274 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.020 1422 0.021) gunicorn-web stdout | 2022-05-19 06:12:20,087 [231] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:20 +0000] "GET /api/v1/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7 HTTP/1.0" 401 274 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" gunicorn-secscan stdout | 2022-05-19 06:12:20,215 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-19 06:12:20,223 [187] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '187'} gunicorn-secscan stdout | 2022-05-19 06:12:20,228 [188] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '188'} nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:20 +0000] "GET /static/img/quay_favicon.png HTTP/1.1" 200 3558 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1311 -) securityworker stdout | 2022-05-19 06:12:20,688 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:12:20,689 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:43.478958+00:00 (in 22.790097 seconds) securityworker stdout | 2022-05-19 06:12:20,689 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:12:50 UTC)" (scheduled at 2022-05-19 06:12:20.688187+00:00) securityworker stdout | 2022-05-19 06:12:20,689 [93] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:12:20,689 [93] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2022-05-19 06:12:20,691 [93] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:12:20,691 [93] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2022-05-19 06:12:20,691 [93] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:12:20,691 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:12:20,691 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:12:20,694 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:12:20,696 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:12:20,706 [93] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:12:20,706 [93] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2022-05-19 06:12:20,706 [93] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2022-05-19 06:12:20,706 [93] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2022-05-19 06:12:20,706 [93] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:12:20,707 [93] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:12:20,707 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:12:20,708 [93] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 46, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 251, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2022-05-19 06:12:20,708 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:12:50 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:12:21,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:12:21,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:31.578357+00:00 (in 9.999261 seconds) chunkcleanupworker stdout | 2022-05-19 06:12:21,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:31 UTC)" (scheduled at 2022-05-19 06:12:21.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:12:21,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:12:21,580 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 12, 21, 579764), True, datetime.datetime(2022, 5, 19, 6, 12, 21, 579764), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:12:21,592 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:12:21,592 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:12:21,592 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:31 UTC)" executed successfully builder stdout | 2022-05-19 06:12:21,856 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:12:21,869 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:12:21,869 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:12:21,877 [56] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-19 06:12:22,429 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'gcworker.py', 'pid': '62'} gunicorn-web stdout | 2022-05-19 06:12:22,485 [230] [DEBUG] [app] Starting request: urn:request:9648fe3b-dd32-4ae5-9827-c63810058668 (/signin) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:22,486 [230] [DEBUG] [app] Ending request: urn:request:9648fe3b-dd32-4ae5-9827-c63810058668 (/signin) {'endpoint': None, 'request_id': 'urn:request:9648fe3b-dd32-4ae5-9827-c63810058668', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin', 'path': '/signin', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:22 +0000] "GET /signin HTTP/1.1" 308 385 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.004 1415 0.003) gunicorn-web stdout | 2022-05-19 06:12:22,487 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:22 +0000] "GET /signin HTTP/1.0" 308 385 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" gunicorn-web stdout | 2022-05-19 06:12:22,817 [235] [DEBUG] [app] Starting request: urn:request:85a54fbe-dbba-44f7-80f1-1a308917038a (/signin/) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:22,823 [235] [DEBUG] [app] Ending request: urn:request:85a54fbe-dbba-44f7-80f1-1a308917038a (/signin/) {'endpoint': 'web.signin', 'request_id': 'urn:request:85a54fbe-dbba-44f7-80f1-1a308917038a', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/', 'path': '/signin/', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:22 +0000] "GET /signin/ HTTP/1.1" 200 13184 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.010 1416 0.009) gunicorn-web stdout | 2022-05-19 06:12:22,825 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:22 +0000] "GET /signin/ HTTP/1.0" 200 84286 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/build/0c928e61-e470-426b-85f5-a2ebeb105ba7" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" namespacegcworker stdout | 2022-05-19 06:12:23,741 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'namespacegcworker.py', 'pid': '71'} servicekey stdout | 2022-05-19 06:12:23,834 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'servicekeyworker.py', 'pid': '94'} nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:24 +0000] "GET /static/ldn/font-awesome.css-a166e3c966c6.css HTTP/1.1" 200 8417 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1218 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:24 +0000] "GET /static/ldn/bootstrap.min.css-13f668494557.css HTTP/1.1" 200 24440 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1219 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:24 +0000] "GET /static/ldn/cssfamilySourceSansPro300400700-30cd793a4b9b.css HTTP/1.1" 200 235 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1233 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:24 +0000] "GET /static/ldn/bootstrap-datetimepicker.min.css-4b15a4ede750.css HTTP/1.1" 200 1482 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1234 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:25 +0000] "GET /static/ldn/cal-heatmap.css-904c3b9e8a60.css HTTP/1.1" 200 799 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1217 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:25 +0000] "GET /static/ldn/ng-tags-input.min.css-7782190f81cd.css HTTP/1.1" 200 1079 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1223 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:25 +0000] "GET /static/build/83-quay-frontend.chunk.js HTTP/1.1" 200 1717 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1198 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:25 +0000] "GET /static/build/135-quay-frontend.chunk.js HTTP/1.1" 200 974 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1199 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:25 +0000] "GET /static/build/18-quay-frontend.chunk.js HTTP/1.1" 200 997 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1198 -) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:25 +0000] "GET /static/build/112-quay-frontend.chunk.js HTTP/1.1" 200 767 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1199 -) gunicorn-web stdout | 2022-05-19 06:12:26,425 [231] [DEBUG] [app] Starting request: urn:request:2ed939b3-efb8-4c8d-b1ec-2a8c044c4246 (/api/v1/user/) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:26,427 [231] [DEBUG] [app] Ending request: urn:request:2ed939b3-efb8-4c8d-b1ec-2a8c044c4246 (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:2ed939b3-efb8-4c8d-b1ec-2a8c044c4246', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:26 +0000] "GET /api/v1/user/ HTTP/1.1" 401 301 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.005 1312 0.006) gunicorn-web stdout | 2022-05-19 06:12:26,429 [231] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:26 +0000] "GET /api/v1/user/ HTTP/1.0" 401 301 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:26 +0000] "GET /static/build/181-quay-frontend.chunk.js HTTP/1.1" 200 951 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1199 -) gunicorn-web stdout | 2022-05-19 06:12:26,431 [235] [DEBUG] [app] Starting request: urn:request:f751169b-18c9-43fd-b19c-ed8dda454afc (/api/v1/messages) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:26,433 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."content", "t1"."uuid", "t1"."severity", "t1"."media_type_id", "t2"."id", "t2"."name" FROM "messages" AS "t1" INNER JOIN "mediatype" AS "t2" ON ("t1"."media_type_id" = "t2"."id")', []) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:26 +0000] "GET /static/build/152-quay-frontend.chunk.js HTTP/1.1" 200 313 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1199 -) gunicorn-web stdout | 2022-05-19 06:12:26,445 [235] [DEBUG] [app] Ending request: urn:request:f751169b-18c9-43fd-b19c-ed8dda454afc (/api/v1/messages) {'endpoint': 'api.globalusermessages', 'request_id': 'urn:request:f751169b-18c9-43fd-b19c-ed8dda454afc', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/messages', 'path': '/api/v1/messages', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:12:26,447 [235] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:26 +0000] "GET /api/v1/messages HTTP/1.1" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.018 1315 0.019) gunicorn-web stdout | 2022-05-19 06:12:26,448 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:26 +0000] "GET /api/v1/messages HTTP/1.0" 200 17 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" repositoryactioncounter stdout | 2022-05-19 06:12:28,057 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositoryactioncounter.py', 'pid': '79'} blobuploadcleanupworker stdout | 2022-05-19 06:12:28,062 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} storagereplication stdout | 2022-05-19 06:12:28,539 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'storagereplication.py', 'pid': '97'} securityworker stdout | 2022-05-19 06:12:28,902 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityworker.py', 'pid': '93'} notificationworker stdout | 2022-05-19 06:12:29,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:12:29,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:39.297880+00:00 (in 9.999338 seconds) notificationworker stdout | 2022-05-19 06:12:29,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:39 UTC)" (scheduled at 2022-05-19 06:12:29.297880+00:00) notificationworker stdout | 2022-05-19 06:12:29,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:12:29,301 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 12, 29, 299420), True, datetime.datetime(2022, 5, 19, 6, 12, 29, 299420), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:12:29,322 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:12:29,322 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:12:29,322 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:39 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:12:29,912 [231] [DEBUG] [app] Starting request: urn:request:cd7b12ac-c6b2-405c-91e6-f33f988fda7d (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:29,913 [235] [DEBUG] [app] Starting request: urn:request:41ac391d-4a6f-4000-9ef1-559f265d15e2 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:29,915 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:12:29,916 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:12:29,918 [241] [DEBUG] [app] Starting request: urn:request:7283348e-4dca-4a3c-9b91-3915e280d9d8 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:12:29,918 [240] [DEBUG] [app] Starting request: urn:request:c2037ed3-85a7-4833-b8ff-bfddafb1ccb8 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:12:29,919 [241] [DEBUG] [app] Ending request: urn:request:7283348e-4dca-4a3c-9b91-3915e280d9d8 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:7283348e-4dca-4a3c-9b91-3915e280d9d8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:12:29,919 [240] [DEBUG] [app] Ending request: urn:request:c2037ed3-85a7-4833-b8ff-bfddafb1ccb8 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c2037ed3-85a7-4833-b8ff-bfddafb1ccb8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:12:29,919 [241] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:12:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2022-05-19 06:12:29,919 [240] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:12:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:12:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) nginx stdout | 127.0.0.1 () - - [19/May/2022:06:12:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2022-05-19 06:12:29,920 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:12:29,920 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:12:29,923 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:12:29,923 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:12:29,925 [231] [DEBUG] [app] Starting request: urn:request:8cc4a281-97df-4184-aff9-170810c8d497 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:12:29,925 [230] [DEBUG] [app] Starting request: urn:request:e5e61c02-224d-47bb-bffa-37d131aca514 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:12:29,925 [231] [DEBUG] [app] Ending request: urn:request:8cc4a281-97df-4184-aff9-170810c8d497 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:8cc4a281-97df-4184-aff9-170810c8d497', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:12:29,926 [230] [DEBUG] [app] Ending request: urn:request:e5e61c02-224d-47bb-bffa-37d131aca514 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e5e61c02-224d-47bb-bffa-37d131aca514', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:12:29,926 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:12:29 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:12:29 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [19/May/2022:06:12:29 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2022-05-19 06:12:29,926 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:12:29,926 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:12:29 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:12:29,927 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:12:29,927 [231] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:12:29,928 [235] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:12:29,934 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:12:29,935 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:12:29,942 [231] [DEBUG] [app] Ending request: urn:request:cd7b12ac-c6b2-405c-91e6-f33f988fda7d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:cd7b12ac-c6b2-405c-91e6-f33f988fda7d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:12:29,942 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:12:29,943 [235] [DEBUG] [app] Ending request: urn:request:41ac391d-4a6f-4000-9ef1-559f265d15e2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:41ac391d-4a6f-4000-9ef1-559f265d15e2', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:12:29,943 [231] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:29 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-19 06:12:29,943 [235] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:29 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.032 118 0.031) gunicorn-web stdout | 2022-05-19 06:12:29,944 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:29 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:29 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.033 118 0.032) repositorygcworker stdout | 2022-05-19 06:12:29,976 [80] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-19 06:12:29,977 [80] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:31.173528+00:00 (in 1.196676 seconds) repositorygcworker stdout | 2022-05-19 06:12:29,977 [80] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:29 UTC)" (scheduled at 2022-05-19 06:12:29.976033+00:00) repositorygcworker stdout | 2022-05-19 06:12:29,977 [80] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-19 06:12:29,979 [80] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 12, 29, 977538), True, datetime.datetime(2022, 5, 19, 6, 12, 29, 977538), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-19 06:12:29,991 [80] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-19 06:12:29,991 [80] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-19 06:12:29,992 [80] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:29 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-19 06:12:30,670 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} queuecleanupworker stdout | 2022-05-19 06:12:30,851 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'queuecleanupworker.py', 'pid': '75'} globalpromstats stdout | 2022-05-19 06:12:30,997 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'globalpromstats.py', 'pid': '63'} repositorygcworker stdout | 2022-05-19 06:12:31,174 [80] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-19 06:12:31,174 [80] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:29.976033+00:00 (in 58.801733 seconds) repositorygcworker stdout | 2022-05-19 06:12:31,174 [80] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:31 UTC)" (scheduled at 2022-05-19 06:12:31.173528+00:00) repositorygcworker stdout | 2022-05-19 06:12:31,175 [80] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-19 06:12:31,175 [80] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:31 UTC)" executed successfully teamsyncworker stdout | 2022-05-19 06:12:31,366 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'teamsyncworker.py', 'pid': '101'} chunkcleanupworker stdout | 2022-05-19 06:12:31,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:12:31,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:41.578357+00:00 (in 9.999330 seconds) chunkcleanupworker stdout | 2022-05-19 06:12:31,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:41 UTC)" (scheduled at 2022-05-19 06:12:31.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:12:31,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:12:31,580 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 12, 31, 579751), True, datetime.datetime(2022, 5, 19, 6, 12, 31, 579751), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:12:31,591 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:12:31,592 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:12:31,592 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:41 UTC)" executed successfully builder stdout | 2022-05-19 06:12:31,888 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:12:31,901 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:12:31,901 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:12:31,910 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:12:32,838 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} expiredappspecifictokenworker stdout | 2022-05-19 06:12:33,132 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} notificationworker stdout | 2022-05-19 06:12:33,308 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'notificationworker.py', 'pid': '73'} builder stdout | 2022-05-19 06:12:33,332 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'builder.py', 'pid': '56'} repositorygcworker stdout | 2022-05-19 06:12:33,610 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositorygcworker.py', 'pid': '80'} gunicorn-web stdout | 2022-05-19 06:12:34,098 [230] [DEBUG] [app] Starting request: urn:request:473971fe-b3f8-4a69-baf0-e74819019dc5 (/api/v1/signin) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:34,103 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) OR ("t1"."email" = %s)) LIMIT %s OFFSET %s', ['whuquay', 'whuquay', 1, 0]) gunicorn-web stdout | 2022-05-19 06:12:34,114 [230] [DEBUG] [app] Ending request: urn:request:473971fe-b3f8-4a69-baf0-e74819019dc5 (/api/v1/signin) {'endpoint': 'api.signin', 'request_id': 'urn:request:473971fe-b3f8-4a69-baf0-e74819019dc5', 'remote_addr': '10.129.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/signin', 'path': '/api/v1/signin', 'parameters': {}, 'json_body': {'username': 'whuquay', 'password': '[FILTERED]'}, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:12:34,115 [230] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:12:34,116 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:34 +0000] "POST /api/v1/signin HTTP/1.0" 403 105 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:34 +0000] "POST /api/v1/signin HTTP/1.1" 403 105 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.020 1534 0.019) manifestbackfillworker stdout | 2022-05-19 06:12:35,998 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} exportactionlogsworker stdout | 2022-05-19 06:12:36,193 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-19 06:12:36,194 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:09.792901+00:00 (in 33.598839 seconds) exportactionlogsworker stdout | 2022-05-19 06:12:36,194 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:36 UTC)" (scheduled at 2022-05-19 06:12:36.193076+00:00) exportactionlogsworker stdout | 2022-05-19 06:12:36,194 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-19 06:12:36,194 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:36 UTC)" executed successfully securityscanningnotificationworker stdout | 2022-05-19 06:12:37,115 [81] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityscanningnotificationworker.py', 'pid': '81'} logrotateworker stdout | 2022-05-19 06:12:37,534 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'logrotateworker.py', 'pid': '68'} gunicorn-web stdout | 2022-05-19 06:12:38,084 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '231'} gunicorn-web stdout | 2022-05-19 06:12:38,096 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '230'} gunicorn-web stdout | 2022-05-19 06:12:38,104 [235] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '235'} gunicorn-web stdout | 2022-05-19 06:12:38,111 [233] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '233'} gunicorn-web stdout | 2022-05-19 06:12:38,128 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-19 06:12:39,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:12:39,299 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:48.405016+00:00 (in 9.106129 seconds) notificationworker stdout | 2022-05-19 06:12:39,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:49 UTC)" (scheduled at 2022-05-19 06:12:39.297880+00:00) notificationworker stdout | 2022-05-19 06:12:39,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:12:39,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 12, 39, 299408), True, datetime.datetime(2022, 5, 19, 6, 12, 39, 299408), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:12:39,319 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:12:39,319 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:12:39,319 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:49 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:12:41,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:12:41,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:51.578357+00:00 (in 9.999230 seconds) chunkcleanupworker stdout | 2022-05-19 06:12:41,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:51 UTC)" (scheduled at 2022-05-19 06:12:41.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:12:41,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:12:41,581 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 12, 41, 579882), True, datetime.datetime(2022, 5, 19, 6, 12, 41, 579882), 0, 'chunk_cleanup/%', 50, 1, 0]) gcworker stdout | 2022-05-19 06:12:41,585 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-19 06:12:41,585 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:11.584428+00:00 (in 29.999242 seconds) gcworker stdout | 2022-05-19 06:12:41,585 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:13:11 UTC)" (scheduled at 2022-05-19 06:12:41.584428+00:00) gcworker stdout | 2022-05-19 06:12:41,585 [62] [DEBUG] [__main__] No GC policies found gcworker stdout | 2022-05-19 06:12:41,586 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:13:11 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:12:41,592 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:12:41,592 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:12:41,592 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:51 UTC)" executed successfully builder stdout | 2022-05-19 06:12:41,920 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:12:41,933 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:12:41,933 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:12:41,942 [56] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-19 06:12:42,637 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-19 06:12:42,646 [234] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '234'} gunicorn-registry stdout | 2022-05-19 06:12:42,651 [238] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '238'} gunicorn-registry stdout | 2022-05-19 06:12:42,663 [232] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '232'} gunicorn-registry stdout | 2022-05-19 06:12:42,668 [241] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '241'} gunicorn-registry stdout | 2022-05-19 06:12:42,675 [240] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '240'} gunicorn-registry stdout | 2022-05-19 06:12:42,686 [236] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '236'} gunicorn-registry stdout | 2022-05-19 06:12:42,692 [239] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '239'} gunicorn-registry stdout | 2022-05-19 06:12:42,698 [237] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '237'} securityworker stdout | 2022-05-19 06:12:43,479 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:12:43,480 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:50.688187+00:00 (in 7.208247 seconds) securityworker stdout | 2022-05-19 06:12:43,480 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:13:13 UTC)" (scheduled at 2022-05-19 06:12:43.478958+00:00) securityworker stdout | 2022-05-19 06:12:43,481 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:12:43,481 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:12:43,484 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:12:43,487 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:12:43,497 [93] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:12:43,499 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:12:43,500 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:13:13 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:12:44,912 [231] [DEBUG] [app] Starting request: urn:request:5d5858f3-72e6-4b70-83f1-80c24ed33c2c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:44,912 [235] [DEBUG] [app] Starting request: urn:request:3663a198-66ee-440c-910d-a433f3c9de34 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:44,914 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:12:44,914 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:12:44,916 [238] [DEBUG] [app] Starting request: urn:request:02aa2997-30c5-4f35-9a9a-fd08070dd4e8 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:12:44,916 [237] [DEBUG] [app] Starting request: urn:request:93802da7-40a9-4559-b4bd-6e4638fd7cd1 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:12:44,917 [238] [DEBUG] [app] Ending request: urn:request:02aa2997-30c5-4f35-9a9a-fd08070dd4e8 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:02aa2997-30c5-4f35-9a9a-fd08070dd4e8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:12:44,917 [238] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:12:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2022-05-19 06:12:44,917 [237] [DEBUG] [app] Ending request: urn:request:93802da7-40a9-4559-b4bd-6e4638fd7cd1 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:93802da7-40a9-4559-b4bd-6e4638fd7cd1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:12:44,918 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [19/May/2022:06:12:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.003) gunicorn-registry stdout | 2022-05-19 06:12:44,918 [237] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:12:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:12:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.003) gunicorn-web stdout | 2022-05-19 06:12:44,918 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:12:44,920 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:12:44,921 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:12:44,922 [235] [DEBUG] [app] Starting request: urn:request:312ccabf-ee81-4e2d-a753-2ae986e9fd07 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:12:44,922 [235] [DEBUG] [app] Ending request: urn:request:312ccabf-ee81-4e2d-a753-2ae986e9fd07 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:312ccabf-ee81-4e2d-a753-2ae986e9fd07', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:12:44,923 [235] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:12:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:12:44,923 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [19/May/2022:06:12:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2022-05-19 06:12:44,923 [230] [DEBUG] [app] Starting request: urn:request:19eddcc2-3ca6-4124-8f92-10625ecd731d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:12:44,924 [235] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:12:44,924 [230] [DEBUG] [app] Ending request: urn:request:19eddcc2-3ca6-4124-8f92-10625ecd731d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:19eddcc2-3ca6-4124-8f92-10625ecd731d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:12:44,925 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:12:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:12:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.003) gunicorn-web stdout | 2022-05-19 06:12:44,925 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:12:44,926 [231] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:12:44,932 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:12:44,933 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:12:44,941 [235] [DEBUG] [app] Ending request: urn:request:3663a198-66ee-440c-910d-a433f3c9de34 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3663a198-66ee-440c-910d-a433f3c9de34', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:12:44,941 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:12:44,941 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:44 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:44 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.030 118 0.031) gunicorn-web stdout | 2022-05-19 06:12:44,942 [231] [DEBUG] [app] Ending request: urn:request:5d5858f3-72e6-4b70-83f1-80c24ed33c2c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5d5858f3-72e6-4b70-83f1-80c24ed33c2c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:12:44,942 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:12:44,942 [231] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:44 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:44 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.031 118 0.032) buildlogsarchiver stdout | 2022-05-19 06:12:45,474 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-19 06:12:45,474 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:15.473591+00:00 (in 29.999241 seconds) buildlogsarchiver stdout | 2022-05-19 06:12:45,474 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:13:15 UTC)" (scheduled at 2022-05-19 06:12:45.473591+00:00) buildlogsarchiver stdout | 2022-05-19 06:12:45,475 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 5, 4, 6, 12, 45, 475113), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-19 06:12:45,487 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-19 06:12:45,487 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-19 06:12:45,487 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:13:15 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-19 06:12:46,438 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} namespacegcworker stdout | 2022-05-19 06:12:46,492 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-19 06:12:46,492 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:16.679426+00:00 (in 30.187026 seconds) namespacegcworker stdout | 2022-05-19 06:12:46,493 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:46 UTC)" (scheduled at 2022-05-19 06:12:46.491684+00:00) namespacegcworker stdout | 2022-05-19 06:12:46,493 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-19 06:12:46,494 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 12, 46, 493837), True, datetime.datetime(2022, 5, 19, 6, 12, 46, 493837), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-19 06:12:46,505 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-19 06:12:46,505 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-19 06:12:46,506 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:46 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:12:46,683 [231] [DEBUG] [app] Starting request: urn:request:94d0c2a0-01db-43a0-9851-a1fbab20e2cd (/api/v1/signin) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:46,687 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) OR ("t1"."email" = %s)) LIMIT %s OFFSET %s', ['whuquay', 'whuquay', 1, 0]) gunicorn-web stdout | 2022-05-19 06:12:46,701 [231] [DEBUG] [app] Ending request: urn:request:94d0c2a0-01db-43a0-9851-a1fbab20e2cd (/api/v1/signin) {'endpoint': 'api.signin', 'request_id': 'urn:request:94d0c2a0-01db-43a0-9851-a1fbab20e2cd', 'remote_addr': '10.129.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/signin', 'path': '/api/v1/signin', 'parameters': {}, 'json_body': {'username': 'whuquay', 'password': '[FILTERED]'}, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:12:46,702 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:12:46,703 [231] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:46 +0000] "POST /api/v1/signin HTTP/1.0" 403 105 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:46 +0000] "POST /api/v1/signin HTTP/1.1" 403 105 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.023 1534 0.022) notificationworker stdout | 2022-05-19 06:12:48,405 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:12:48,406 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:49.297880+00:00 (in 0.892107 seconds) notificationworker stdout | 2022-05-19 06:12:48,406 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:48 UTC)" (scheduled at 2022-05-19 06:12:48.405016+00:00) notificationworker stdout | 2022-05-19 06:12:48,406 [73] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-19 06:12:48,406 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:48 UTC)" executed successfully notificationworker stdout | 2022-05-19 06:12:49,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:12:49,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:59.297880+00:00 (in 9.999174 seconds) notificationworker stdout | 2022-05-19 06:12:49,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:59 UTC)" (scheduled at 2022-05-19 06:12:49.297880+00:00) notificationworker stdout | 2022-05-19 06:12:49,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:12:49,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 12, 49, 299403), True, datetime.datetime(2022, 5, 19, 6, 12, 49, 299403), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:12:49,312 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:12:49,312 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:12:49,312 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:12:59 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-19 06:12:50,230 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-19 06:12:50,237 [187] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '187'} gunicorn-secscan stdout | 2022-05-19 06:12:50,242 [188] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '188'} securityworker stdout | 2022-05-19 06:12:50,688 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:12:50,689 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:13.478958+00:00 (in 22.789901 seconds) securityworker stdout | 2022-05-19 06:12:50,689 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:13:20 UTC)" (scheduled at 2022-05-19 06:12:50.688187+00:00) securityworker stdout | 2022-05-19 06:12:50,689 [93] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:12:50,689 [93] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2022-05-19 06:12:50,691 [93] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:12:50,691 [93] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2022-05-19 06:12:50,691 [93] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:12:50,691 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:12:50,692 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:12:50,695 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:12:50,698 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:12:50,711 [93] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:12:50,711 [93] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2022-05-19 06:12:50,711 [93] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2022-05-19 06:12:50,711 [93] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2022-05-19 06:12:50,712 [93] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:12:50,713 [93] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:12:50,713 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:12:50,713 [93] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 46, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 251, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2022-05-19 06:12:50,714 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:13:20 UTC)" executed successfully securityscanningnotificationworker stdout | 2022-05-19 06:12:51,479 [81] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-19 06:12:51,480 [81] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:07.583673+00:00 (in 16.103616 seconds) securityscanningnotificationworker stdout | 2022-05-19 06:12:51,480 [81] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:51 UTC)" (scheduled at 2022-05-19 06:12:51.479280+00:00) securityscanningnotificationworker stdout | 2022-05-19 06:12:51,480 [81] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-19 06:12:51,481 [81] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 12, 51, 480806), True, datetime.datetime(2022, 5, 19, 6, 12, 51, 480806), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-19 06:12:51,492 [81] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-19 06:12:51,492 [81] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-19 06:12:51,492 [81] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:51 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:12:51,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:12:51,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:12:59.780399+00:00 (in 8.201272 seconds) chunkcleanupworker stdout | 2022-05-19 06:12:51,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:01 UTC)" (scheduled at 2022-05-19 06:12:51.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:12:51,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:12:51,580 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 12, 51, 579684), True, datetime.datetime(2022, 5, 19, 6, 12, 51, 579684), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:12:51,591 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:12:51,591 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:12:51,591 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:01 UTC)" executed successfully builder stdout | 2022-05-19 06:12:51,952 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:12:51,965 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:12:51,966 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:12:51,976 [56] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-19 06:12:52,454 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'gcworker.py', 'pid': '62'} namespacegcworker stdout | 2022-05-19 06:12:53,756 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'namespacegcworker.py', 'pid': '71'} servicekey stdout | 2022-05-19 06:12:53,871 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'servicekeyworker.py', 'pid': '94'} blobuploadcleanupworker stdout | 2022-05-19 06:12:58,090 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} repositoryactioncounter stdout | 2022-05-19 06:12:58,098 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositoryactioncounter.py', 'pid': '79'} storagereplication stdout | 2022-05-19 06:12:58,567 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'storagereplication.py', 'pid': '97'} securityworker stdout | 2022-05-19 06:12:58,944 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityworker.py', 'pid': '93'} notificationworker stdout | 2022-05-19 06:12:59,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:12:59,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:09.297880+00:00 (in 9.999159 seconds) notificationworker stdout | 2022-05-19 06:12:59,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:09 UTC)" (scheduled at 2022-05-19 06:12:59.297880+00:00) notificationworker stdout | 2022-05-19 06:12:59,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:12:59,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 12, 59, 299363), True, datetime.datetime(2022, 5, 19, 6, 12, 59, 299363), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:12:59,319 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:12:59,320 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:12:59,320 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:09 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:12:59,780 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:12:59,781 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:01.578357+00:00 (in 1.797302 seconds) chunkcleanupworker stdout | 2022-05-19 06:12:59,781 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:59 UTC)" (scheduled at 2022-05-19 06:12:59.780399+00:00) chunkcleanupworker stdout | 2022-05-19 06:12:59,781 [58] [DEBUG] [workers.queueworker] Running watchdog. chunkcleanupworker stdout | 2022-05-19 06:12:59,781 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:13:59 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:12:59,913 [230] [DEBUG] [app] Starting request: urn:request:ac673190-3201-47f6-90cd-e2bd4c45aa3e (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:59,913 [233] [DEBUG] [app] Starting request: urn:request:260edda9-2444-4731-92e0-e1aa7023781c (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:12:59,915 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:12:59,916 [233] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:12:59,918 [238] [DEBUG] [app] Starting request: urn:request:154a64bb-ca35-43f9-b466-916d0fa78b1f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:12:59,918 [241] [DEBUG] [app] Starting request: urn:request:206100d0-53ce-4c86-bac3-c66e058c3837 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:12:59,918 [238] [DEBUG] [app] Ending request: urn:request:154a64bb-ca35-43f9-b466-916d0fa78b1f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:154a64bb-ca35-43f9-b466-916d0fa78b1f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:12:59,919 [241] [DEBUG] [app] Ending request: urn:request:206100d0-53ce-4c86-bac3-c66e058c3837 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:206100d0-53ce-4c86-bac3-c66e058c3837', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:12:59,919 [238] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:12:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2022-05-19 06:12:59,919 [241] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:12:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:12:59,919 [233] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:12:59,919 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [19/May/2022:06:12:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [19/May/2022:06:12:59 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2022-05-19 06:12:59,921 [233] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:12:59,922 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:12:59,923 [233] [DEBUG] [app] Starting request: urn:request:9cf58968-70df-466e-8812-64e67fd0378b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:12:59,924 [233] [DEBUG] [app] Ending request: urn:request:9cf58968-70df-466e-8812-64e67fd0378b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9cf58968-70df-466e-8812-64e67fd0378b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:12:59,924 [235] [DEBUG] [app] Starting request: urn:request:a08c35a8-aa9e-4bb9-bb26-0d7029e9929d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:12:59,924 [233] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:12:59 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:12:59,925 [235] [DEBUG] [app] Ending request: urn:request:a08c35a8-aa9e-4bb9-bb26-0d7029e9929d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a08c35a8-aa9e-4bb9-bb26-0d7029e9929d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:12:59 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.001 159 0.002) gunicorn-web stdout | 2022-05-19 06:12:59,925 [233] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:12:59,925 [235] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:12:59 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:12:59 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2022-05-19 06:12:59,925 [233] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:12:59,926 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:12:59,927 [230] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:12:59,932 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:12:59,933 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:12:59,941 [233] [DEBUG] [app] Ending request: urn:request:260edda9-2444-4731-92e0-e1aa7023781c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:260edda9-2444-4731-92e0-e1aa7023781c', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:12:59,941 [230] [DEBUG] [app] Ending request: urn:request:ac673190-3201-47f6-90cd-e2bd4c45aa3e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ac673190-3201-47f6-90cd-e2bd4c45aa3e', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:12:59,941 [233] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:12:59,941 [230] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:12:59,942 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:59 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-19 06:12:59,942 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:12:59 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:59 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.031 118 0.030) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:12:59 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.031 118 0.030) buildlogsarchiver stdout | 2022-05-19 06:13:00,687 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} queuecleanupworker stdout | 2022-05-19 06:13:00,884 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'queuecleanupworker.py', 'pid': '75'} globalpromstats stdout | 2022-05-19 06:13:01,025 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'globalpromstats.py', 'pid': '63'} teamsyncworker stdout | 2022-05-19 06:13:01,407 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'teamsyncworker.py', 'pid': '101'} chunkcleanupworker stdout | 2022-05-19 06:13:01,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:13:01,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:11.578357+00:00 (in 9.999207 seconds) chunkcleanupworker stdout | 2022-05-19 06:13:01,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:11 UTC)" (scheduled at 2022-05-19 06:13:01.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:13:01,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:13:01,580 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 13, 1, 579829), True, datetime.datetime(2022, 5, 19, 6, 13, 1, 579829), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:13:01,594 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:13:01,594 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:13:01,594 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:11 UTC)" executed successfully builder stdout | 2022-05-19 06:13:01,987 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:13:02,001 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:13:02,001 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:13:02,010 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:13:02,868 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} expiredappspecifictokenworker stdout | 2022-05-19 06:13:03,167 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} notificationworker stdout | 2022-05-19 06:13:03,325 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'notificationworker.py', 'pid': '73'} builder stdout | 2022-05-19 06:13:03,370 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'builder.py', 'pid': '56'} repositorygcworker stdout | 2022-05-19 06:13:03,653 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositorygcworker.py', 'pid': '80'} manifestbackfillworker stdout | 2022-05-19 06:13:06,030 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} securityscanningnotificationworker stdout | 2022-05-19 06:13:07,154 [81] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityscanningnotificationworker.py', 'pid': '81'} logrotateworker stdout | 2022-05-19 06:13:07,561 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'logrotateworker.py', 'pid': '68'} securityscanningnotificationworker stdout | 2022-05-19 06:13:07,584 [81] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-19 06:13:07,584 [81] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:42.583285+00:00 (in 34.998886 seconds) securityscanningnotificationworker stdout | 2022-05-19 06:13:07,585 [81] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:14:07 UTC)" (scheduled at 2022-05-19 06:13:07.583673+00:00) securityscanningnotificationworker stdout | 2022-05-19 06:13:07,585 [81] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-19 06:13:07,585 [81] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:14:07 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:13:08,100 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '231'} gunicorn-web stdout | 2022-05-19 06:13:08,111 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '230'} gunicorn-web stdout | 2022-05-19 06:13:08,119 [235] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '235'} gunicorn-web stdout | 2022-05-19 06:13:08,126 [233] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '233'} gunicorn-web stdout | 2022-05-19 06:13:08,143 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-19 06:13:09,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:13:09,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:19.297880+00:00 (in 9.999150 seconds) notificationworker stdout | 2022-05-19 06:13:09,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:19 UTC)" (scheduled at 2022-05-19 06:13:09.297880+00:00) notificationworker stdout | 2022-05-19 06:13:09,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:13:09,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 13, 9, 299335), True, datetime.datetime(2022, 5, 19, 6, 13, 9, 299335), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:13:09,344 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:13:09,344 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:13:09,345 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:19 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-19 06:13:09,793 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-19 06:13:09,793 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:36.193076+00:00 (in 26.399477 seconds) exportactionlogsworker stdout | 2022-05-19 06:13:09,794 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:14:09 UTC)" (scheduled at 2022-05-19 06:13:09.792901+00:00) exportactionlogsworker stdout | 2022-05-19 06:13:09,794 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-19 06:13:09,795 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 13, 9, 794229), True, datetime.datetime(2022, 5, 19, 6, 13, 9, 794229), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-19 06:13:09,810 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-19 06:13:09,810 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-19 06:13:09,810 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:14:09 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:13:11,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:13:11,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:21.578357+00:00 (in 9.999195 seconds) chunkcleanupworker stdout | 2022-05-19 06:13:11,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:21 UTC)" (scheduled at 2022-05-19 06:13:11.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:13:11,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:13:11,581 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 13, 11, 580036), True, datetime.datetime(2022, 5, 19, 6, 13, 11, 580036), 0, 'chunk_cleanup/%', 50, 1, 0]) gcworker stdout | 2022-05-19 06:13:11,584 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-19 06:13:11,585 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:41.584428+00:00 (in 29.999251 seconds) gcworker stdout | 2022-05-19 06:13:11,585 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:13:41 UTC)" (scheduled at 2022-05-19 06:13:11.584428+00:00) gcworker stdout | 2022-05-19 06:13:11,586 [62] [DEBUG] [__main__] No GC policies found gcworker stdout | 2022-05-19 06:13:11,587 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:13:41 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:13:11,594 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:13:11,594 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:13:11,594 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:21 UTC)" executed successfully builder stdout | 2022-05-19 06:13:12,020 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:13:12,035 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:13:12,035 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:13:12,048 [56] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-19 06:13:12,656 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-19 06:13:12,671 [238] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '238'} gunicorn-registry stdout | 2022-05-19 06:13:12,671 [234] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '234'} gunicorn-registry stdout | 2022-05-19 06:13:12,682 [232] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '232'} gunicorn-registry stdout | 2022-05-19 06:13:12,689 [241] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '241'} gunicorn-registry stdout | 2022-05-19 06:13:12,703 [240] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '240'} gunicorn-registry stdout | 2022-05-19 06:13:12,719 [239] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '239'} gunicorn-registry stdout | 2022-05-19 06:13:12,726 [236] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '236'} gunicorn-registry stdout | 2022-05-19 06:13:12,733 [237] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '237'} securityworker stdout | 2022-05-19 06:13:13,479 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:13:13,479 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:20.688187+00:00 (in 7.208521 seconds) securityworker stdout | 2022-05-19 06:13:13,480 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:13:43 UTC)" (scheduled at 2022-05-19 06:13:13.478958+00:00) securityworker stdout | 2022-05-19 06:13:13,480 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:13:13,480 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:13:13,483 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:13:13,485 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:13:13,495 [93] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:13:13,497 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:13:13,498 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:13:43 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:13:14,912 [230] [DEBUG] [app] Starting request: urn:request:ada5624f-8f81-4472-9c4c-46108919cc84 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:14,912 [235] [DEBUG] [app] Starting request: urn:request:13526e8e-6817-416a-ac35-75ab9bf41dfe (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:14,914 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:13:14,914 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:13:14,917 [232] [DEBUG] [app] Starting request: urn:request:ceb2a88c-eb0c-4cb3-8087-35bd5a3b71d1 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:13:14,918 [241] [DEBUG] [app] Starting request: urn:request:2d870969-32ef-453b-8d46-650d0889ccb3 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:13:14,918 [232] [DEBUG] [app] Ending request: urn:request:ceb2a88c-eb0c-4cb3-8087-35bd5a3b71d1 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ceb2a88c-eb0c-4cb3-8087-35bd5a3b71d1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:13:14,918 [232] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:13:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:13:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.003) gunicorn-web stdout | 2022-05-19 06:13:14,918 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-19 06:13:14,918 [241] [DEBUG] [app] Ending request: urn:request:2d870969-32ef-453b-8d46-650d0889ccb3 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2d870969-32ef-453b-8d46-650d0889ccb3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:13:14,920 [241] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:13:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:13:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.004 162 0.003) gunicorn-web stdout | 2022-05-19 06:13:14,920 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:13:14,921 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:13:14,923 [233] [DEBUG] [app] Starting request: urn:request:4598df89-cca9-49c0-8b9b-962661f9a278 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:13:14,924 [233] [DEBUG] [app] Ending request: urn:request:4598df89-cca9-49c0-8b9b-962661f9a278 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4598df89-cca9-49c0-8b9b-962661f9a278', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:13:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.002) gunicorn-web stdout | 2022-05-19 06:13:14,925 [233] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:13:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:13:14,925 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:13:14,926 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:13:14,926 [235] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:13:14,929 [233] [DEBUG] [app] Starting request: urn:request:a2b231e1-eb01-4bf2-bb60-9b24d5618e94 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:13:14,929 [233] [DEBUG] [app] Ending request: urn:request:a2b231e1-eb01-4bf2-bb60-9b24d5618e94 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a2b231e1-eb01-4bf2-bb60-9b24d5618e94', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:13:14,930 [233] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:13:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:13:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.003) gunicorn-web stdout | 2022-05-19 06:13:14,932 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:13:14,934 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:13:14,935 [230] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:13:14,944 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:13:14,946 [235] [DEBUG] [app] Ending request: urn:request:13526e8e-6817-416a-ac35-75ab9bf41dfe (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:13526e8e-6817-416a-ac35-75ab9bf41dfe', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:13:14,946 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:13:14,946 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:14 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:14 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.035 118 0.036) gunicorn-web stdout | 2022-05-19 06:13:14,955 [230] [DEBUG] [app] Ending request: urn:request:ada5624f-8f81-4472-9c4c-46108919cc84 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ada5624f-8f81-4472-9c4c-46108919cc84', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:13:14,955 [230] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:13:14,956 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:14 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:14 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.045 118 0.045) buildlogsarchiver stdout | 2022-05-19 06:13:15,474 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-19 06:13:15,474 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:45.473591+00:00 (in 29.999081 seconds) buildlogsarchiver stdout | 2022-05-19 06:13:15,475 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:13:45 UTC)" (scheduled at 2022-05-19 06:13:15.473591+00:00) buildlogsarchiver stdout | 2022-05-19 06:13:15,477 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 5, 4, 6, 13, 15, 475530), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-19 06:13:15,492 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-19 06:13:15,492 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-19 06:13:15,492 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:13:45 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-19 06:13:16,478 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} namespacegcworker stdout | 2022-05-19 06:13:16,680 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-19 06:13:16,680 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:21.678921+00:00 (in 4.998677 seconds) namespacegcworker stdout | 2022-05-19 06:13:16,680 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:14:16 UTC)" (scheduled at 2022-05-19 06:13:16.679426+00:00) namespacegcworker stdout | 2022-05-19 06:13:16,681 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-19 06:13:16,681 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:14:16 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:13:16,706 [233] [DEBUG] [app] Starting request: urn:request:0175ec90-dd71-45b7-aa56-90ef220c96c3 (/api/v1/user/) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:16,709 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ((("t1"."username" = %s) AND ("t1"."organization" = %s)) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:16,722 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) OR ("t1"."email" = %s)) LIMIT %s OFFSET %s', ['whuquay', 'bogus@bogus.com', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:16,726 [233] [DEBUG] [data.model.user] Email and username are unique! gunicorn-web stdout | 2022-05-19 06:13:16,728 [233] [DEBUG] [peewee] ('INSERT INTO "user" ("uuid", "username", "email", "verified", "organization", "robot", "invoice_email", "invalid_login_attempts", "last_invalid_login", "removed_tag_expiration_s", "enabled", "maximum_queued_builds_count", "creation_date") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "user"."id"', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', 'whuquay', 'bogus@bogus.com', False, False, False, False, 0, datetime.datetime(2022, 5, 19, 6, 13, 16, 727541), 1209600, True, None, datetime.datetime(2022, 5, 19, 6, 13, 16, 727561)]) gunicorn-web stdout | 2022-05-19 06:13:17,021 [233] [DEBUG] [peewee] ('UPDATE "user" SET "uuid" = %s, "username" = %s, "password_hash" = %s, "email" = %s, "verified" = %s, "organization" = %s, "robot" = %s, "invoice_email" = %s, "invalid_login_attempts" = %s, "last_invalid_login" = %s, "removed_tag_expiration_s" = %s, "enabled" = %s, "maximum_queued_builds_count" = %s, "creation_date" = %s WHERE ("user"."id" = %s)', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', 'whuquay', '$2b$12$kVpC2snsN5Im/cGIHPVoGO2K3mnIfjO8wUmlmkL7/2HzhT61Qbh9i', 'bogus@bogus.com', True, False, False, False, 0, datetime.datetime(2022, 5, 19, 6, 13, 16, 727541), 1209600, True, None, datetime.datetime(2022, 5, 19, 6, 13, 16, 727561), 1]) gunicorn-web stdout | 2022-05-19 06:13:17,028 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:17,031 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:17,034 [233] [DEBUG] [endpoints.common] Successfully signed in as user whuquay with uuid ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:17,034 [233] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:17,034 [233] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:17,036 [233] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'whuquay']) gunicorn-web stdout | 2022-05-19 06:13:17,044 [233] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:17,046 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:17,049 [233] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:17,049 [233] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:17,050 [233] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:17,050 [233] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2022-05-19 06:13:17,051 [233] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 1]) gunicorn-web stdout | 2022-05-19 06:13:17,056 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) gunicorn-web stdout | 2022-05-19 06:13:17,061 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:17,064 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2022-05-19 06:13:17,068 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:17,074 [233] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."repository_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."repository_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."repository_id" = "t1"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1652940797071, 0, 1, None]) gunicorn-web stdout | 2022-05-19 06:13:17,081 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:17,085 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2022-05-19 06:13:17,088 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:17,092 [233] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."repository_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."repository_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."repository_id" = "t2"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1652940797091, 0, 1]) gunicorn-web stdout | 2022-05-19 06:13:17,094 [233] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:17,095 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:17,098 [233] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:17,099 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:17,103 [233] [DEBUG] [app] Ending request: urn:request:0175ec90-dd71-45b7-aa56-90ef220c96c3 (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:0175ec90-dd71-45b7-aa56-90ef220c96c3', 'remote_addr': '10.129.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': {'username': 'whuquay', 'email': 'bogus@bogus.com', 'password': '[FILTERED]', 'repeatPassword': '12345678'}, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:17,104 [233] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:13:17,105 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:17 +0000] "POST /api/v1/user/ HTTP/1.0" 200 663 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:17 +0000] "POST /api/v1/user/ HTTP/1.1" 200 663 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.402 1568 0.402) gunicorn-web stdout | 2022-05-19 06:13:17,669 [230] [DEBUG] [app] Starting request: urn:request:37057c1b-66b9-4f34-8c0a-bd366b94534d (/api/v1/user/) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:17,669 [230] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:17,669 [230] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:17,671 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:17,681 [230] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:17,681 [230] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:17,681 [230] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:17,682 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:17,682 [230] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:17,682 [230] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:17,682 [230] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:17,682 [230] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2022-05-19 06:13:17,683 [230] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'whuquay']) gunicorn-web stdout | 2022-05-19 06:13:17,688 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:17,689 [230] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 1]) gunicorn-web stdout | 2022-05-19 06:13:17,692 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) gunicorn-web stdout | 2022-05-19 06:13:17,695 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:17,698 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2022-05-19 06:13:17,700 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:17,704 [230] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."repository_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."repository_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."repository_id" = "t1"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1652940797702, 0, 1, None]) gunicorn-web stdout | 2022-05-19 06:13:17,708 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:17,710 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2022-05-19 06:13:17,713 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:17,715 [230] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."repository_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."repository_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."repository_id" = "t2"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1652940797715, 0, 1]) gunicorn-web stdout | 2022-05-19 06:13:17,718 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:17,718 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:17,718 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:17,719 [230] [DEBUG] [app] Ending request: urn:request:37057c1b-66b9-4f34-8c0a-bd366b94534d (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:37057c1b-66b9-4f34-8c0a-bd366b94534d', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:17,721 [230] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:17 +0000] "GET /api/v1/user/ HTTP/1.1" 200 660 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.054 1566 0.054) gunicorn-web stdout | 2022-05-19 06:13:17,721 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:17 +0000] "GET /api/v1/user/ HTTP/1.0" 200 660 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/signin/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" gunicorn-web stdout | 2022-05-19 06:13:18,734 [235] [DEBUG] [app] Starting request: urn:request:d7b54728-2f7d-4fcc-8ddf-a381d48596db (/api/v1/user/notifications) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:18,735 [235] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:18,735 [230] [DEBUG] [app] Starting request: urn:request:45b7b35f-89b5-49de-a35a-2359639d273b (/api/v1/user/apptoken) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:18,735 [235] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:18,735 [230] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:18,735 [230] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:18,736 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:18,736 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:18,749 [235] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:18,750 [235] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:18,750 [235] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:18,750 [230] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:18,750 [235] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2022-05-19 06:13:18,750 [235] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:18,750 [235] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:18,751 [235] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:18,751 [235] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:18,751 [230] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:18,751 [230] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:18,751 [235] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2022-05-19 06:13:18,751 [230] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2022-05-19 06:13:18,751 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:18,752 [230] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:18,752 [230] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:18,752 [230] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:18,752 [230] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2022-05-19 06:13:18,753 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2022, 5, 22, 1, 25, 18, 752850), datetime.datetime(2022, 5, 19, 6, 13, 18, 752982)]) gunicorn-web stdout | 2022-05-19 06:13:18,753 [235] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2022-05-19 06:13:18,761 [230] [DEBUG] [app] Ending request: urn:request:45b7b35f-89b5-49de-a35a-2359639d273b (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:45b7b35f-89b5-49de-a35a-2359639d273b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:18,762 [235] [DEBUG] [app] Ending request: urn:request:d7b54728-2f7d-4fcc-8ddf-a381d48596db (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:d7b54728-2f7d-4fcc-8ddf-a381d48596db', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:18,763 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:13:18,763 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:18 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:18 +0000] "GET /api/v1/user/notifications HTTP/1.1" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.030 1572 0.031) gunicorn-web stdout | 2022-05-19 06:13:18,763 [230] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:18 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.1" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.031 1581 0.031) gunicorn-web stdout | 2022-05-19 06:13:18,765 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:18 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" notificationworker stdout | 2022-05-19 06:13:19,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:13:19,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:29.297880+00:00 (in 9.999165 seconds) notificationworker stdout | 2022-05-19 06:13:19,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:29 UTC)" (scheduled at 2022-05-19 06:13:19.297880+00:00) notificationworker stdout | 2022-05-19 06:13:19,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:13:19,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 13, 19, 299638), True, datetime.datetime(2022, 5, 19, 6, 13, 19, 299638), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:13:19,313 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:13:19,313 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:13:19,313 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:29 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-19 06:13:20,245 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-19 06:13:20,252 [187] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '187'} gunicorn-secscan stdout | 2022-05-19 06:13:20,265 [188] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'secscan:application', 'pid': '188'} securityworker stdout | 2022-05-19 06:13:20,688 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:13:20,689 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:43.478958+00:00 (in 22.790002 seconds) securityworker stdout | 2022-05-19 06:13:20,689 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:13:50 UTC)" (scheduled at 2022-05-19 06:13:20.688187+00:00) securityworker stdout | 2022-05-19 06:13:20,689 [93] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:13:20,689 [93] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2022-05-19 06:13:20,691 [93] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:13:20,692 [93] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2022-05-19 06:13:20,692 [93] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:13:20,692 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:13:20,692 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:13:20,695 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:13:20,697 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:13:20,707 [93] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:13:20,707 [93] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2022-05-19 06:13:20,707 [93] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2022-05-19 06:13:20,707 [93] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2022-05-19 06:13:20,708 [93] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-05-19 06:13:20,708 [93] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-05-19 06:13:20,709 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:13:20,709 [93] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 46, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 251, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2022-05-19 06:13:20,709 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:13:50 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:13:20,728 [233] [DEBUG] [app] Starting request: urn:request:1bc072bf-6c0b-4ba3-8705-9bf3614e62a6 (/api/v1/repository) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:20,728 [233] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:20,729 [233] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:20,730 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:20,740 [233] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:20,741 [233] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:20,741 [233] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:20,743 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-web stdout | 2022-05-19 06:13:20,748 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2022-05-19 06:13:20,755 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2022-05-19 06:13:20,761 [233] [DEBUG] [app] Ending request: urn:request:1bc072bf-6c0b-4ba3-8705-9bf3614e62a6 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:1bc072bf-6c0b-4ba3-8705-9bf3614e62a6', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/repository?last_modified=true&popularity=true&starred=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'popularity': 'true', 'starred': 'true'}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:20,762 [233] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:20 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/1.1" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.037 1623 0.036) gunicorn-web stdout | 2022-05-19 06:13:20,763 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:20 +0000] "GET /api/v1/repository?last_modified=true&popularity=true&starred=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" gunicorn-web stdout | 2022-05-19 06:13:21,481 [233] [DEBUG] [app] Starting request: urn:request:da89e122-d87f-419e-ad4b-128355e87147 (/api/v1/repository) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:21,481 [233] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:21,481 [233] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:21,482 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:21,493 [233] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:21,493 [233] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:21,493 [233] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:21,495 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['whuquay', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:21,500 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1"', []) gunicorn-web stdout | 2022-05-19 06:13:21,505 [233] [DEBUG] [peewee] ('((SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE (((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) AND ("t3"."user_id" = %s))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t7" ON ("t6"."team_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t8"."team_id" = "t7"."id") WHERE (((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t8"."user_id" = %s)))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t9" ON ("t9"."organization_id" = "t4"."namespace_user_id") INNER JOIN "teammember" AS "t10" ON ("t9"."id" = "t10"."team_id") WHERE ((((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t9"."role_id" = %s)) AND ("t10"."user_id" = %s))) ORDER BY rid LIMIT %s', [3, 1, 'whuquay', 1, 3, 1, 'whuquay', 1, 3, 1, 'whuquay', 1, 1, 101]) gunicorn-web stdout | 2022-05-19 06:13:21,515 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2022-05-19 06:13:21,521 [233] [DEBUG] [app] Ending request: urn:request:da89e122-d87f-419e-ad4b-128355e87147 (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:da89e122-d87f-419e-ad4b-128355e87147', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'whuquay', 'popularity': 'true', 'quota': 'true', 'state': 'true'}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:21,522 [233] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:21 +0000] "GET /api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true HTTP/1.1" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.044 1650 0.043) gunicorn-web stdout | 2022-05-19 06:13:21,523 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:21 +0000] "GET /api/v1/repository?last_modified=true&namespace=whuquay&popularity=true"a=true&state=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" chunkcleanupworker stdout | 2022-05-19 06:13:21,579 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:13:21,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:31.578357+00:00 (in 9.999155 seconds) chunkcleanupworker stdout | 2022-05-19 06:13:21,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:31 UTC)" (scheduled at 2022-05-19 06:13:21.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:13:21,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:13:21,580 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 13, 21, 579893), True, datetime.datetime(2022, 5, 19, 6, 13, 21, 579893), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:13:21,593 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:13:21,593 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:13:21,594 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:31 UTC)" executed successfully namespacegcworker stdout | 2022-05-19 06:13:21,679 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-19 06:13:21,679 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:46.491684+00:00 (in 24.811931 seconds) namespacegcworker stdout | 2022-05-19 06:13:21,680 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-19 06:18:21 UTC)" (scheduled at 2022-05-19 06:13:21.678921+00:00) namespacegcworker stdout | 2022-05-19 06:13:21,681 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 19, 6, 13, 21, 680465), 'namespacegc/%']) namespacegcworker stdout | 2022-05-19 06:13:21,693 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 19, 6, 13, 21, 680465), True, datetime.datetime(2022, 5, 19, 6, 13, 21, 680465), 0, 'namespacegc/%']) namespacegcworker stdout | 2022-05-19 06:13:21,697 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 19, 6, 13, 21, 680465), True, datetime.datetime(2022, 5, 19, 6, 13, 21, 680465), 0, 'namespacegc/%', False, datetime.datetime(2022, 5, 19, 6, 13, 21, 680465), 'namespacegc/%']) namespacegcworker stdout | 2022-05-19 06:13:21,700 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-19 06:13:21,701 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-19 06:18:21 UTC)" executed successfully builder stdout | 2022-05-19 06:13:22,049 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:13:22,063 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:13:22,063 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:13:22,072 [56] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-19 06:13:22,495 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'gcworker.py', 'pid': '62'} namespacegcworker stdout | 2022-05-19 06:13:23,794 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'namespacegcworker.py', 'pid': '71'} servicekey stdout | 2022-05-19 06:13:23,885 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'servicekeyworker.py', 'pid': '94'} blobuploadcleanupworker stdout | 2022-05-19 06:13:28,130 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} repositoryactioncounter stdout | 2022-05-19 06:13:28,137 [79] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositoryactioncounter.py', 'pid': '79'} storagereplication stdout | 2022-05-19 06:13:28,597 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'storagereplication.py', 'pid': '97'} securityworker stdout | 2022-05-19 06:13:28,991 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityworker.py', 'pid': '93'} notificationworker stdout | 2022-05-19 06:13:29,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:13:29,299 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:39.297880+00:00 (in 9.998906 seconds) notificationworker stdout | 2022-05-19 06:13:29,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:39 UTC)" (scheduled at 2022-05-19 06:13:29.297880+00:00) notificationworker stdout | 2022-05-19 06:13:29,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:13:29,301 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 13, 29, 299924), True, datetime.datetime(2022, 5, 19, 6, 13, 29, 299924), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:13:29,313 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:13:29,313 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:13:29,314 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:39 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:13:29,913 [231] [DEBUG] [app] Starting request: urn:request:57cfe267-9978-4413-99d8-5e5abe889325 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:29,913 [235] [DEBUG] [app] Starting request: urn:request:70b03cca-0dc0-42b0-923b-a6b6cf1d0c97 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:29,915 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:13:29,915 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:13:29,918 [240] [DEBUG] [app] Starting request: urn:request:c62ece6e-fdfd-4bce-8855-8c3042038c0f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:13:29,918 [237] [DEBUG] [app] Starting request: urn:request:de8ad0cb-a0f4-4c7e-b319-489037e97d1b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:13:29,919 [240] [DEBUG] [app] Ending request: urn:request:c62ece6e-fdfd-4bce-8855-8c3042038c0f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c62ece6e-fdfd-4bce-8855-8c3042038c0f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:13:29,919 [237] [DEBUG] [app] Ending request: urn:request:de8ad0cb-a0f4-4c7e-b319-489037e97d1b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:de8ad0cb-a0f4-4c7e-b319-489037e97d1b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:13:29,919 [240] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:13:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:13:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 162 0.002) gunicorn-web stdout | 2022-05-19 06:13:29,920 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [19/May/2022:06:13:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.005 162 0.005) gunicorn-registry stdout | 2022-05-19 06:13:29,922 [237] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:13:29 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:13:29,922 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:13:29,922 [235] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:13:29,925 [231] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:13:29,925 [235] [DEBUG] [app] Starting request: urn:request:50f964cd-2eb9-4d36-b1e4-ff7807652ed2 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:13:29,925 [235] [DEBUG] [app] Ending request: urn:request:50f964cd-2eb9-4d36-b1e4-ff7807652ed2 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:50f964cd-2eb9-4d36-b1e4-ff7807652ed2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:13:29 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.002) gunicorn-web stdout | 2022-05-19 06:13:29,926 [235] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:13:29 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:13:29,927 [235] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:13:29,927 [231] [DEBUG] [app] Starting request: urn:request:0d6d7d92-2cb3-426e-830b-ad74623cfcf5 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:13:29,927 [235] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:13:29,928 [231] [DEBUG] [app] Ending request: urn:request:0d6d7d92-2cb3-426e-830b-ad74623cfcf5 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0d6d7d92-2cb3-426e-830b-ad74623cfcf5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:13:29,928 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:13:29 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:13:29 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.003) gunicorn-web stdout | 2022-05-19 06:13:29,929 [231] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:13:29,929 [231] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:13:29,936 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:13:29,936 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:13:29,948 [235] [DEBUG] [app] Ending request: urn:request:70b03cca-0dc0-42b0-923b-a6b6cf1d0c97 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:70b03cca-0dc0-42b0-923b-a6b6cf1d0c97', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:13:29,948 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:13:29,948 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:29 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:29 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.037 118 0.037) gunicorn-web stdout | 2022-05-19 06:13:29,950 [231] [DEBUG] [app] Ending request: urn:request:57cfe267-9978-4413-99d8-5e5abe889325 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:57cfe267-9978-4413-99d8-5e5abe889325', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:13:29,950 [231] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:29 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.040 118 0.039) gunicorn-web stdout | 2022-05-19 06:13:29,951 [231] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:29 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" repositorygcworker stdout | 2022-05-19 06:13:29,976 [80] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-19 06:13:29,977 [80] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:31.173528+00:00 (in 1.196327 seconds) repositorygcworker stdout | 2022-05-19 06:13:29,977 [80] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:14:29 UTC)" (scheduled at 2022-05-19 06:13:29.976033+00:00) repositorygcworker stdout | 2022-05-19 06:13:29,978 [80] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-19 06:13:29,979 [80] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 13, 29, 978302), True, datetime.datetime(2022, 5, 19, 6, 13, 29, 978302), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-19 06:13:29,994 [80] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-19 06:13:29,995 [80] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-19 06:13:29,995 [80] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:14:29 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:13:30,565 [235] [DEBUG] [app] Starting request: urn:request:caf6c4e7-a4d7-4b23-883a-1a3dc35bfff6 (/api/v1/organization/) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:30,565 [235] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:30,566 [235] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:30,567 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:30,579 [235] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:30,579 [235] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:30,579 [235] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:30,580 [235] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2022-05-19 06:13:30,581 [235] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:30,581 [235] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:30,581 [235] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:30,581 [235] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:30,581 [235] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2022-05-19 06:13:30,585 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'build_test', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:30,590 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['build_test', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:30,595 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) OR ("t1"."email" = %s)) LIMIT %s OFFSET %s', ['build_test', 'e0e1443c-f196-4604-b9f6-e949ce59eb83', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:30,598 [235] [DEBUG] [data.model.user] Email and username are unique! gunicorn-web stdout | 2022-05-19 06:13:30,600 [235] [DEBUG] [peewee] ('INSERT INTO "user" ("uuid", "username", "email", "verified", "organization", "robot", "invoice_email", "invalid_login_attempts", "last_invalid_login", "removed_tag_expiration_s", "enabled", "maximum_queued_builds_count", "creation_date") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "user"."id"', ['409d92ed-5842-4b42-a297-d11f278e9327', 'build_test', 'e0e1443c-f196-4604-b9f6-e949ce59eb83', False, False, False, False, 0, datetime.datetime(2022, 5, 19, 6, 13, 30, 598788), 1209600, True, None, datetime.datetime(2022, 5, 19, 6, 13, 30, 598791)]) gunicorn-web stdout | 2022-05-19 06:13:30,602 [235] [DEBUG] [peewee] ('UPDATE "user" SET "uuid" = %s, "username" = %s, "email" = %s, "verified" = %s, "organization" = %s, "robot" = %s, "invoice_email" = %s, "invalid_login_attempts" = %s, "last_invalid_login" = %s, "removed_tag_expiration_s" = %s, "enabled" = %s, "maximum_queued_builds_count" = %s, "creation_date" = %s WHERE ("user"."id" = %s)', ['409d92ed-5842-4b42-a297-d11f278e9327', 'build_test', 'e0e1443c-f196-4604-b9f6-e949ce59eb83', False, True, False, False, 0, datetime.datetime(2022, 5, 19, 6, 13, 30, 598788), 1209600, True, None, datetime.datetime(2022, 5, 19, 6, 13, 30, 598791), 2]) gunicorn-web stdout | 2022-05-19 06:13:30,604 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['admin', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:30,606 [235] [DEBUG] [peewee] ('INSERT INTO "team" ("name", "organization_id", "role_id", "description") VALUES (%s, %s, %s, %s) RETURNING "team"."id"', ['owners', 2, 1, '']) gunicorn-web stdout | 2022-05-19 06:13:30,609 [235] [DEBUG] [peewee] ('INSERT INTO "teammember" ("user_id", "team_id") VALUES (%s, %s) RETURNING "teammember"."id"', [1, 1]) gunicorn-web stdout | 2022-05-19 06:13:30,617 [235] [DEBUG] [app] Ending request: urn:request:caf6c4e7-a4d7-4b23-883a-1a3dc35bfff6 (/api/v1/organization/) {'endpoint': 'api.organizationlist', 'request_id': 'urn:request:caf6c4e7-a4d7-4b23-883a-1a3dc35bfff6', 'remote_addr': '10.129.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/organization/', 'path': '/api/v1/organization/', 'parameters': {}, 'json_body': {'name': 'build_test'}, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:30,618 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:13:30,619 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:30 +0000] "POST /api/v1/organization/ HTTP/1.0" 201 10 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organizations/new/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:30 +0000] "POST /api/v1/organization/ HTTP/1.1" 201 10 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organizations/new/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.056 1764 0.055) buildlogsarchiver stdout | 2022-05-19 06:13:30,718 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} queuecleanupworker stdout | 2022-05-19 06:13:30,899 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'queuecleanupworker.py', 'pid': '75'} globalpromstats stdout | 2022-05-19 06:13:31,058 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'globalpromstats.py', 'pid': '63'} repositorygcworker stdout | 2022-05-19 06:13:31,174 [80] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-19 06:13:31,174 [80] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:14:29.976033+00:00 (in 58.801417 seconds) repositorygcworker stdout | 2022-05-19 06:13:31,175 [80] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:14:31 UTC)" (scheduled at 2022-05-19 06:13:31.173528+00:00) repositorygcworker stdout | 2022-05-19 06:13:31,175 [80] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-19 06:13:31,175 [80] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:14:31 UTC)" executed successfully teamsyncworker stdout | 2022-05-19 06:13:31,431 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'teamsyncworker.py', 'pid': '101'} chunkcleanupworker stdout | 2022-05-19 06:13:31,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:13:31,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:41.578357+00:00 (in 9.999169 seconds) chunkcleanupworker stdout | 2022-05-19 06:13:31,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:41 UTC)" (scheduled at 2022-05-19 06:13:31.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:13:31,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:13:31,580 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 13, 31, 579828), True, datetime.datetime(2022, 5, 19, 6, 13, 31, 579828), 0, 'chunk_cleanup/%', 50, 1, 0]) chunkcleanupworker stdout | 2022-05-19 06:13:31,595 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:13:31,595 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:13:31,595 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:41 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:13:31,755 [230] [DEBUG] [app] Starting request: urn:request:b1dd6fd0-ae6c-4142-98b8-0882c8224311 (/api/v1/user/) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:31,755 [230] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,755 [230] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,756 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:31,766 [230] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:31,767 [230] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:31,767 [230] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,767 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,767 [230] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:31,767 [230] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:31,767 [230] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:31,767 [230] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2022-05-19 06:13:31,769 [230] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" INNER JOIN "team" AS "t2" ON ("t2"."organization_id" = "t1"."id") INNER JOIN "teammember" AS "t3" ON ("t3"."team_id" = "t2"."id") INNER JOIN "user" AS "t4" ON ("t4"."id" = "t3"."user_id") WHERE (("t1"."organization" = %s) AND ("t4"."username" = %s))', [True, 'whuquay']) gunicorn-web stdout | 2022-05-19 06:13:31,774 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,774 [230] [DEBUG] [peewee] ('SELECT "t1"."service_ident", "t2"."name", "t1"."metadata_json" FROM "federatedlogin" AS "t1" INNER JOIN "loginservice" AS "t2" ON ("t1"."service_id" = "t2"."id") WHERE (("t2"."name" != %s) AND ("t1"."user_id" = %s))', ['quayrobot', 1]) gunicorn-web stdout | 2022-05-19 06:13:31,777 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."kind_id" FROM "userprompt" AS "t1" INNER JOIN "userpromptkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) gunicorn-web stdout | 2022-05-19 06:13:31,783 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:31,786 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2022-05-19 06:13:31,789 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:31,792 [230] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."repository_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."repository_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."repository_id" = "t1"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1652940811791, 0, 1, None]) gunicorn-web stdout | 2022-05-19 06:13:31,797 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:31,802 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['whuquay']) gunicorn-web stdout | 2022-05-19 06:13:31,805 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['whuquay', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:31,808 [230] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."repository_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."repository_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."repository_id" = "t2"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1652940811807, 0, 1]) gunicorn-web stdout | 2022-05-19 06:13:31,810 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,811 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,811 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,813 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2022-05-19 06:13:31,817 [230] [DEBUG] [auth.permissions] Organization team added permission: namespacewide(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2022-05-19 06:13:31,817 [230] [DEBUG] [auth.permissions] Organization team added repo permission: namespacewide(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2022-05-19 06:13:31,817 [230] [DEBUG] [auth.permissions] Team added permission: teamwideneed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2022-05-19 06:13:31,818 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,818 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,819 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,822 [230] [DEBUG] [app] Ending request: urn:request:b1dd6fd0-ae6c-4142-98b8-0882c8224311 (/api/v1/user/) {'endpoint': 'api.user', 'request_id': 'urn:request:b1dd6fd0-ae6c-4142-98b8-0882c8224311', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/user/', 'path': '/api/v1/user/', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:31,823 [230] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:31 +0000] "GET /api/v1/user/ HTTP/1.1" 200 921 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organizations/new/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.071 1596 0.072) gunicorn-web stdout | 2022-05-19 06:13:31,825 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:31 +0000] "GET /api/v1/user/ HTTP/1.0" 200 921 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organizations/new/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" gunicorn-web stdout | 2022-05-19 06:13:31,858 [230] [DEBUG] [app] Starting request: urn:request:9dd5d8f6-b2df-4986-83b0-631b51397ebb (/api/v1/organization/build_test) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:31,859 [230] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,859 [230] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,859 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:31,870 [230] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:31,870 [230] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:31,870 [230] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,872 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'build_test', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:31,874 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,874 [230] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:31,874 [230] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:31,874 [230] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:31,874 [230] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2022-05-19 06:13:31,877 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2022-05-19 06:13:31,882 [230] [DEBUG] [auth.permissions] Organization team added permission: namespacewide(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2022-05-19 06:13:31,882 [230] [DEBUG] [auth.permissions] Organization team added repo permission: namespacewide(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2022-05-19 06:13:31,883 [230] [DEBUG] [auth.permissions] Team added permission: teamwideneed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2022-05-19 06:13:31,884 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t2" ON ("t1"."role_id" = "t2"."id") WHERE ("t1"."organization_id" = %s)', [2]) gunicorn-web stdout | 2022-05-19 06:13:31,886 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1"', []) gunicorn-web stdout | 2022-05-19 06:13:31,889 [230] [DEBUG] [peewee] ('SELECT "t1"."team_id", Count("t1"."id") FROM "repositorypermission" AS "t1" WHERE ("t1"."team_id" IN (%s)) GROUP BY "t1"."team_id"', [1]) gunicorn-web stdout | 2022-05-19 06:13:31,892 [230] [DEBUG] [peewee] ('SELECT "t1"."team_id", Count("t1"."id") FROM "teammember" AS "t1" WHERE ("t1"."team_id" IN (%s)) GROUP BY "t1"."team_id"', [1]) gunicorn-web stdout | 2022-05-19 06:13:31,894 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,895 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,895 [230] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:31,896 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-web stdout | 2022-05-19 06:13:31,900 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['build_test', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:31,903 [230] [DEBUG] [peewee] ('INSERT INTO "repositorysize" ("repository_id", "size_bytes") SELECT "t1"."id", sum("t2"."layers_compressed_size") FROM "manifest" AS "t2" INNER JOIN "repository" AS "t1" ON ("t2"."repository_id" = "t1"."id") INNER JOIN (SELECT "t3"."repository_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."repository_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."repository_id" = "t1"."id") LEFT OUTER JOIN "repositorysize" AS "repositorysize" ON ("repositorysize"."repository_id" = "t1"."id") WHERE (("t1"."namespace_user_id" = %s) AND ("repositorysize"."repository_id" IS %s)) GROUP BY "t1"."id" RETURNING "repositorysize"."id"', [False, None, 1652940811902, 0, 2, None]) gunicorn-web stdout | 2022-05-19 06:13:31,908 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['build_test', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:31,911 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-web stdout | 2022-05-19 06:13:31,913 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', ['build_test', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:31,917 [230] [DEBUG] [peewee] ('SELECT sum("t1"."layers_compressed_size") FROM "manifest" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN (SELECT "t3"."repository_id" FROM "tag" AS "t3" WHERE (("t3"."hidden" = %s) AND (("t3"."lifetime_end_ms" IS %s) OR ("t3"."lifetime_end_ms" > %s))) GROUP BY "t3"."repository_id" HAVING (Count("t3"."name") > %s)) AS "t4" ON ("t4"."repository_id" = "t2"."id") WHERE ("t2"."namespace_user_id" = %s)', [False, None, 1652940811916, 0, 2]) gunicorn-web stdout | 2022-05-19 06:13:31,921 [230] [DEBUG] [app] Ending request: urn:request:9dd5d8f6-b2df-4986-83b0-631b51397ebb (/api/v1/organization/build_test) {'endpoint': 'api.organization', 'request_id': 'urn:request:9dd5d8f6-b2df-4986-83b0-631b51397ebb', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/organization/build_test', 'path': '/api/v1/organization/build_test', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:31,922 [230] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:31 +0000] "GET /api/v1/organization/build_test HTTP/1.1" 200 762 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organization/build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.067 1619 0.066) gunicorn-web stdout | 2022-05-19 06:13:31,923 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:31 +0000] "GET /api/v1/organization/build_test HTTP/1.0" 200 762 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organization/build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" builder stdout | 2022-05-19 06:13:32,083 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:13:32,097 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:13:32,097 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:13:32,106 [56] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:13:32,898 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} expiredappspecifictokenworker stdout | 2022-05-19 06:13:33,195 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} notificationworker stdout | 2022-05-19 06:13:33,342 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'notificationworker.py', 'pid': '73'} builder stdout | 2022-05-19 06:13:33,411 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'builder.py', 'pid': '56'} repositorygcworker stdout | 2022-05-19 06:13:33,698 [80] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'repositorygcworker.py', 'pid': '80'} gunicorn-web stdout | 2022-05-19 06:13:34,943 [235] [DEBUG] [app] Starting request: urn:request:ac688c5a-8147-42db-bf83-31f97109cce5 (/api/v1/organization/build_test/proxycache) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:34,943 [235] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:34,943 [235] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:34,944 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:34,957 [235] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:34,957 [235] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:34,957 [235] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:34,958 [235] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:34,958 [235] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:34,958 [235] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:34,959 [235] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:34,959 [235] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2022-05-19 06:13:34,962 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2022-05-19 06:13:34,972 [235] [DEBUG] [auth.permissions] Organization team added permission: namespacewide(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2022-05-19 06:13:34,972 [235] [DEBUG] [auth.permissions] Organization team added repo permission: namespacewide(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2022-05-19 06:13:34,972 [235] [DEBUG] [auth.permissions] Team added permission: teamwideneed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2022-05-19 06:13:34,974 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."organization_id", "t1"."creation_date", "t1"."upstream_registry", "t1"."upstream_registry_username", "t1"."upstream_registry_password", "t1"."expiration_s", "t1"."insecure" FROM "proxycacheconfig" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t2"."organization" = %s)) LIMIT %s OFFSET %s', ['build_test', True, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:34,980 [235] [DEBUG] [app] Ending request: urn:request:ac688c5a-8147-42db-bf83-31f97109cce5 (/api/v1/organization/build_test/proxycache) {'endpoint': 'api.organizationproxycacheconfig', 'request_id': 'urn:request:ac688c5a-8147-42db-bf83-31f97109cce5', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/organization/build_test/proxycache', 'path': '/api/v1/organization/build_test/proxycache', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:34,982 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:13:34,983 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:34 +0000] "GET /api/v1/organization/build_test/proxycache HTTP/1.0" 200 62 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organization/build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:34 +0000] "GET /api/v1/organization/build_test/proxycache HTTP/1.1" 200 62 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organization/build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.042 1611 0.043) gunicorn-web stdout | 2022-05-19 06:13:35,545 [235] [DEBUG] [app] Starting request: urn:request:bcd0b181-ac5e-448b-8694-52603ae0e0c9 (/api/v1/organization/build_test/prototypes) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:35,546 [235] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:35,546 [235] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:35,547 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:35,558 [235] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:35,558 [235] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:35,558 [235] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:35,559 [235] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:35,559 [235] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:35,559 [235] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:35,559 [235] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:35,559 [235] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2022-05-19 06:13:35,561 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2022-05-19 06:13:35,565 [235] [DEBUG] [auth.permissions] Organization team added permission: namespacewide(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2022-05-19 06:13:35,565 [235] [DEBUG] [auth.permissions] Organization team added repo permission: namespacewide(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2022-05-19 06:13:35,565 [235] [DEBUG] [auth.permissions] Team added permission: teamwideneed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2022-05-19 06:13:35,566 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'build_test', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:35,569 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" LEFT OUTER JOIN "user" AS "t2" ON ("t2"."id" = "t1"."activating_user_id") LEFT OUTER JOIN "user" AS "t3" ON ("t3"."id" = "t1"."delegate_user_id") LEFT OUTER JOIN "team" AS "t4" ON ("t4"."id" = "t1"."delegate_team_id") LEFT OUTER JOIN "role" AS "t5" ON ("t5"."id" = "t1"."role_id") WHERE ("t1"."org_id" = %s)', [2]) gunicorn-web stdout | 2022-05-19 06:13:35,573 [235] [DEBUG] [app] Ending request: urn:request:bcd0b181-ac5e-448b-8694-52603ae0e0c9 (/api/v1/organization/build_test/prototypes) {'endpoint': 'api.permissionprototypelist', 'request_id': 'urn:request:bcd0b181-ac5e-448b-8694-52603ae0e0c9', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/organization/build_test/prototypes', 'path': '/api/v1/organization/build_test/prototypes', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:35,574 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:13:35,575 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:35 +0000] "GET /api/v1/organization/build_test/prototypes HTTP/1.0" 200 19 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organization/build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:35 +0000] "GET /api/v1/organization/build_test/prototypes HTTP/1.1" 200 19 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organization/build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.031 1611 0.031) manifestbackfillworker stdout | 2022-05-19 06:13:36,067 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} exportactionlogsworker stdout | 2022-05-19 06:13:36,193 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-19 06:13:36,194 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:14:09.792901+00:00 (in 33.598970 seconds) exportactionlogsworker stdout | 2022-05-19 06:13:36,194 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:14:36 UTC)" (scheduled at 2022-05-19 06:13:36.193076+00:00) exportactionlogsworker stdout | 2022-05-19 06:13:36,194 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-19 06:13:36,194 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:14:36 UTC)" executed successfully nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:36 +0000] "GET /static/img/creditcards/credit.png HTTP/1.1" 200 1776 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organization/build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.000 1523 -) gunicorn-web stdout | 2022-05-19 06:13:36,353 [235] [DEBUG] [app] Starting request: urn:request:47ebb189-7566-48b7-a494-334361f9edb7 (/api/v1/user/apptoken) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:36,353 [235] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:36,353 [235] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:36,353 [231] [DEBUG] [app] Starting request: urn:request:4eaa0c85-6c01-4e62-bf2c-10288c3974bc (/api/v1/organization/build_test/quota) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:36,354 [231] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:36,354 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:36,354 [231] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:36,355 [233] [DEBUG] [app] Starting request: urn:request:dd56ebae-381c-4c2e-b487-2ef5f31715a0 (/api/v1/user/notifications) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:36,355 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:36,355 [233] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:36,355 [233] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:36,356 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:36,358 [230] [DEBUG] [app] Starting request: urn:request:6068653b-af79-4a77-8131-1de3c2220b0b (/api/v1/repository) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:36,359 [230] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:36,359 [230] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:36,360 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:36,364 [235] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:36,365 [235] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:36,365 [235] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:36,365 [233] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:36,365 [235] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2022-05-19 06:13:36,365 [235] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:36,365 [233] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:36,365 [235] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:36,365 [233] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:36,365 [235] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:36,365 [235] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:36,365 [235] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2022-05-19 06:13:36,365 [233] [DEBUG] [endpoints.api] Checking permission for user whuquay gunicorn-web stdout | 2022-05-19 06:13:36,365 [233] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:36,365 [233] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:36,365 [233] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:36,365 [231] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:36,366 [233] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:36,366 [233] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2022-05-19 06:13:36,366 [231] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:36,366 [231] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:36,366 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2022, 5, 22, 1, 25, 36, 365929), datetime.datetime(2022, 5, 19, 6, 13, 36, 366036)]) gunicorn-web stdout | 2022-05-19 06:13:36,367 [231] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:36,367 [231] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:36,367 [231] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:36,367 [231] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:36,367 [231] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2022-05-19 06:13:36,368 [233] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2022-05-19 06:13:36,370 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2022-05-19 06:13:36,371 [235] [DEBUG] [app] Ending request: urn:request:47ebb189-7566-48b7-a494-334361f9edb7 (/api/v1/user/apptoken) {'endpoint': 'api.apptokens', 'request_id': 'urn:request:47ebb189-7566-48b7-a494-334361f9edb7', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/user/apptoken?expiring=true', 'path': '/api/v1/user/apptoken', 'parameters': {'expiring': 'true'}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:36,372 [230] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:36,372 [230] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:36,372 [230] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:36,372 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:13:36,373 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:36 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organization/build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:36 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.1" 200 38 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organization/build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.022 1604 0.021) gunicorn-web stdout | 2022-05-19 06:13:36,374 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['whuquay', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:36,374 [231] [DEBUG] [auth.permissions] Organization team added permission: namespacewide(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2022-05-19 06:13:36,375 [231] [DEBUG] [auth.permissions] Organization team added repo permission: namespacewide(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2022-05-19 06:13:36,375 [231] [DEBUG] [auth.permissions] Team added permission: teamwideneed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2022-05-19 06:13:36,376 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."organization" = %s) AND ("t1"."username" = %s)) LIMIT %s OFFSET %s', [True, 'build_test', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:36,377 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-web stdout | 2022-05-19 06:13:36,377 [233] [DEBUG] [app] Ending request: urn:request:dd56ebae-381c-4c2e-b487-2ef5f31715a0 (/api/v1/user/notifications) {'endpoint': 'api.usernotificationlist', 'request_id': 'urn:request:dd56ebae-381c-4c2e-b487-2ef5f31715a0', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/user/notifications', 'path': '/api/v1/user/notifications', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:36,378 [233] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:36 +0000] "GET /api/v1/user/notifications HTTP/1.1" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organization/build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.025 1595 0.025) gunicorn-web stdout | 2022-05-19 06:13:36,379 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:36 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organization/build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" gunicorn-web stdout | 2022-05-19 06:13:36,379 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."limit_bytes" FROM "userorganizationquota" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_id" = "t2"."id") WHERE ("t2"."username" = %s)', ['build_test']) gunicorn-web stdout | 2022-05-19 06:13:36,380 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['public', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:36,383 [231] [DEBUG] [app] Ending request: urn:request:4eaa0c85-6c01-4e62-bf2c-10288c3974bc (/api/v1/organization/build_test/quota) {'endpoint': 'api.organizationquotalist', 'request_id': 'urn:request:4eaa0c85-6c01-4e62-bf2c-10288c3974bc', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/organization/build_test/quota', 'path': '/api/v1/organization/build_test/quota', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:36,384 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:13:36,385 [231] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:36 +0000] "GET /api/v1/organization/build_test/quota HTTP/1.0" 200 3 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organization/build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:36 +0000] "GET /api/v1/organization/build_test/quota HTTP/1.1" 200 3 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organization/build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.034 1606 0.033) gunicorn-web stdout | 2022-05-19 06:13:36,385 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1"', []) gunicorn-web stdout | 2022-05-19 06:13:36,395 [230] [DEBUG] [peewee] ('(((SELECT DISTINCT "t1"."name", "t1"."id" AS "rid", "t1"."description", "t2"."username", "t1"."visibility_id", "t1"."kind_id", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") LEFT OUTER JOIN "repositorypermission" AS "t3" ON ("t3"."repository_id" = "t1"."id") WHERE (((("t1"."state" != %s) AND ("t1"."kind_id" = %s)) AND ("t2"."username" = %s)) AND ("t1"."visibility_id" = %s))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") WHERE (((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t6"."user_id" = %s)))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t7" ON ("t6"."team_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t8"."team_id" = "t7"."id") WHERE (((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t8"."user_id" = %s)))) UNION (SELECT DISTINCT "t4"."name", "t4"."id" AS "rid", "t4"."description", "t5"."username", "t4"."visibility_id", "t4"."kind_id", "t4"."state" FROM "repository" AS "t4" INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "repositorypermission" AS "t6" ON ("t6"."repository_id" = "t4"."id") INNER JOIN "team" AS "t9" ON ("t9"."organization_id" = "t4"."namespace_user_id") INNER JOIN "teammember" AS "t10" ON ("t9"."id" = "t10"."team_id") WHERE ((((("t4"."state" != %s) AND ("t4"."kind_id" = %s)) AND ("t5"."username" = %s)) AND ("t9"."role_id" = %s)) AND ("t10"."user_id" = %s))) ORDER BY rid LIMIT %s', [3, 1, 'build_test', 1, 3, 1, 'build_test', 1, 3, 1, 'build_test', 1, 3, 1, 'build_test', 1, 1, 101]) gunicorn-web stdout | 2022-05-19 06:13:36,410 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name", "t1"."id" AS "rid" FROM "repository" AS "t1" INNER JOIN "star" AS "t4" ON ("t4"."repository_id" = "t1"."id") INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE ((("t4"."user_id" = %s) AND ("t1"."kind_id" = %s)) AND ("t1"."state" != %s))', [1, 1, 3]) gunicorn-web stdout | 2022-05-19 06:13:36,416 [230] [DEBUG] [app] Ending request: urn:request:6068653b-af79-4a77-8131-1de3c2220b0b (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:6068653b-af79-4a77-8131-1de3c2220b0b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=true"a=true', 'path': '/api/v1/repository', 'parameters': {'last_modified': 'true', 'namespace': 'build_test', 'popularity': 'true', 'public': 'true', 'quota': 'true'}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:36,421 [230] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:36 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=true"a=true HTTP/1.1" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organization/build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.065 1666 0.066) gunicorn-web stdout | 2022-05-19 06:13:36,423 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:36 +0000] "GET /api/v1/repository?last_modified=true&namespace=build_test&popularity=true&public=true"a=true HTTP/1.0" 200 21 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/organization/build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" securityscanningnotificationworker stdout | 2022-05-19 06:13:37,174 [81] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'securityscanningnotificationworker.py', 'pid': '81'} logrotateworker stdout | 2022-05-19 06:13:37,581 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'logrotateworker.py', 'pid': '68'} gunicorn-web stdout | 2022-05-19 06:13:38,118 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '231'} gunicorn-web stdout | 2022-05-19 06:13:38,126 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '230'} gunicorn-web stdout | 2022-05-19 06:13:38,140 [235] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '235'} gunicorn-web stdout | 2022-05-19 06:13:38,146 [233] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '233'} gunicorn-web stdout | 2022-05-19 06:13:38,156 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-19 06:13:39,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:13:39,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:48.405016+00:00 (in 9.106338 seconds) notificationworker stdout | 2022-05-19 06:13:39,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:49 UTC)" (scheduled at 2022-05-19 06:13:39.297880+00:00) notificationworker stdout | 2022-05-19 06:13:39,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:13:39,300 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 13, 39, 299302), True, datetime.datetime(2022, 5, 19, 6, 13, 39, 299302), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:13:39,318 [73] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-19 06:13:39,319 [73] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-19 06:13:39,319 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:49 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:13:41,578 [58] [DEBUG] [apscheduler.scheduler] Looking for jobs to run chunkcleanupworker stdout | 2022-05-19 06:13:41,579 [58] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:51.578357+00:00 (in 9.999343 seconds) chunkcleanupworker stdout | 2022-05-19 06:13:41,579 [58] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:51 UTC)" (scheduled at 2022-05-19 06:13:41.578357+00:00) chunkcleanupworker stdout | 2022-05-19 06:13:41,579 [58] [DEBUG] [workers.queueworker] Getting work item from queue. chunkcleanupworker stdout | 2022-05-19 06:13:41,580 [58] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 13, 41, 579625), True, datetime.datetime(2022, 5, 19, 6, 13, 41, 579625), 0, 'chunk_cleanup/%', 50, 1, 0]) gcworker stdout | 2022-05-19 06:13:41,585 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-19 06:13:41,585 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:14:11.584428+00:00 (in 29.999098 seconds) gcworker stdout | 2022-05-19 06:13:41,585 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:14:11 UTC)" (scheduled at 2022-05-19 06:13:41.584428+00:00) gcworker stdout | 2022-05-19 06:13:41,586 [62] [DEBUG] [__main__] No GC policies found gcworker stdout | 2022-05-19 06:13:41,586 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-19 06:14:11 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-19 06:13:41,592 [58] [DEBUG] [workers.queueworker] No more work. chunkcleanupworker stdout | 2022-05-19 06:13:41,592 [58] [DEBUG] [data.database] Disconnecting from database. chunkcleanupworker stdout | 2022-05-19 06:13:41,592 [58] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:51 UTC)" executed successfully builder stdout | 2022-05-19 06:13:42,116 [56] [DEBUG] [buildman.manager.ephemeral] Checking for more work from the build queue builder stdout | 2022-05-19 06:13:42,136 [56] [DEBUG] [buildman.manager.ephemeral] No additional work found. Going to sleep for 10 seconds builder stdout | 2022-05-19 06:13:42,136 [56] [DEBUG] [buildman.manager.ephemeral] Writing queue metrics builder stdout | 2022-05-19 06:13:42,144 [56] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-19 06:13:42,584 [81] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-19 06:13:42,584 [81] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:51.479280+00:00 (in 8.895076 seconds) securityscanningnotificationworker stdout | 2022-05-19 06:13:42,584 [81] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-19 06:18:42 UTC)" (scheduled at 2022-05-19 06:13:42.583285+00:00) securityscanningnotificationworker stdout | 2022-05-19 06:13:42,586 [81] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 19, 6, 13, 42, 584952), 'secscanv4/%']) securityscanningnotificationworker stdout | 2022-05-19 06:13:42,599 [81] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 19, 6, 13, 42, 584952), True, datetime.datetime(2022, 5, 19, 6, 13, 42, 584952), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2022-05-19 06:13:42,604 [81] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 19, 6, 13, 42, 584952), True, datetime.datetime(2022, 5, 19, 6, 13, 42, 584952), 0, 'secscanv4/%', False, datetime.datetime(2022, 5, 19, 6, 13, 42, 584952), 'secscanv4/%']) securityscanningnotificationworker stdout | 2022-05-19 06:13:42,607 [81] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-19 06:13:42,608 [81] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-19 06:18:42 UTC)" executed successfully gunicorn-registry stdout | 2022-05-19 06:13:42,668 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-19 06:13:42,687 [234] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '234'} gunicorn-registry stdout | 2022-05-19 06:13:42,692 [238] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '238'} gunicorn-registry stdout | 2022-05-19 06:13:42,710 [241] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '241'} gunicorn-registry stdout | 2022-05-19 06:13:42,717 [232] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '232'} gunicorn-registry stdout | 2022-05-19 06:13:42,725 [240] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '240'} gunicorn-registry stdout | 2022-05-19 06:13:42,734 [239] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '239'} gunicorn-registry stdout | 2022-05-19 06:13:42,752 [237] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '237'} gunicorn-registry stdout | 2022-05-19 06:13:42,760 [236] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'registry:application', 'pid': '236'} securityworker stdout | 2022-05-19 06:13:43,479 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-19 06:13:43,479 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:50.688187+00:00 (in 7.208531 seconds) securityworker stdout | 2022-05-19 06:13:43,480 [93] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:14:13 UTC)" (scheduled at 2022-05-19 06:13:43.478958+00:00) securityworker stdout | 2022-05-19 06:13:43,480 [93] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-19 06:13:43,480 [93] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-05-19 06:13:43,483 [93] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-19 06:13:43,485 [93] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:13:43,493 [93] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-19 06:13:43,495 [93] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-19 06:13:43,495 [93] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-19 06:14:13 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:13:44,912 [230] [DEBUG] [app] Starting request: urn:request:bbdfc55d-2520-4d3a-8a60-982821f3d7a6 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:44,912 [233] [DEBUG] [app] Starting request: urn:request:46b1e167-7015-4ae4-b3a4-73f370ee7ed4 (/health/instance) {'X-Forwarded-For': '10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:44,915 [233] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:13:44,915 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-19 06:13:44,918 [232] [DEBUG] [app] Starting request: urn:request:8504cfbe-d94e-45bc-a29d-c47b658d4eed (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-05-19 06:13:44,919 [232] [DEBUG] [app] Ending request: urn:request:8504cfbe-d94e-45bc-a29d-c47b658d4eed (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8504cfbe-d94e-45bc-a29d-c47b658d4eed', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [19/May/2022:06:13:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.003) gunicorn-registry stdout | 2022-05-19 06:13:44,920 [232] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:13:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" gunicorn-registry stdout | 2022-05-19 06:13:44,920 [234] [DEBUG] [app] Starting request: urn:request:52e55387-49b8-4e3a-b4d5-100168c2fb34 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:13:44,920 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-19 06:13:44,921 [234] [DEBUG] [app] Ending request: urn:request:52e55387-49b8-4e3a-b4d5-100168c2fb34 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:52e55387-49b8-4e3a-b4d5-100168c2fb34', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '46f1d72d', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-05-19 06:13:44,922 [234] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:13:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:13:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 162 0.003) gunicorn-web stdout | 2022-05-19 06:13:44,922 [233] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:13:44,922 [230] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:13:44,925 [235] [DEBUG] [app] Starting request: urn:request:96970694-2463-4f3b-b2c2-66c6e8b7db39 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:13:44,925 [235] [DEBUG] [app] Ending request: urn:request:96970694-2463-4f3b-b2c2-66c6e8b7db39 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:96970694-2463-4f3b-b2c2-66c6e8b7db39', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:13:44,926 [235] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:13:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [19/May/2022:06:13:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.002 159 0.002) gunicorn-web stdout | 2022-05-19 06:13:44,926 [230] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:13:44,927 [230] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:13:44,928 [233] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-19 06:13:44,933 [233] [DEBUG] [app] Starting request: urn:request:eae1618b-f382-45e5-b6e3-c775ef047a9e (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-05-19 06:13:44,934 [233] [DEBUG] [app] Ending request: urn:request:eae1618b-f382-45e5-b6e3-c775ef047a9e (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:eae1618b-f382-45e5-b6e3-c775ef047a9e', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'python-requests/2.27.1'} gunicorn-web stdout | 2022-05-19 06:13:44,935 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) nginx stdout | 127.0.0.1 () - - [19/May/2022:06:13:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.006 159 0.006) gunicorn-web stdout | 2022-05-19 06:13:44,938 [233] [INFO] [gunicorn.access] 127.0.0.1 - - [19/May/2022:06:13:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-05-19 06:13:44,939 [233] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-19 06:13:44,940 [233] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-19 06:13:44,947 [230] [DEBUG] [app] Ending request: urn:request:bbdfc55d-2520-4d3a-8a60-982821f3d7a6 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:bbdfc55d-2520-4d3a-8a60-982821f3d7a6', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:13:44,947 [230] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:13:44,947 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-19 06:13:44,949 [230] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:44 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:44 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.037 118 0.038) gunicorn-web stdout | 2022-05-19 06:13:44,959 [233] [DEBUG] [app] Ending request: urn:request:46b1e167-7015-4ae4-b3a4-73f370ee7ed4 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:46b1e167-7015-4ae4-b3a4-73f370ee7ed4', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'http://10.129.2.84/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'kube-probe/1.22'} gunicorn-web stdout | 2022-05-19 06:13:44,959 [233] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:13:44,960 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:44 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:44 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.049 118 0.049) buildlogsarchiver stdout | 2022-05-19 06:13:45,474 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-19 06:13:45,474 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:14:15.473591+00:00 (in 29.999144 seconds) buildlogsarchiver stdout | 2022-05-19 06:13:45,475 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:14:15 UTC)" (scheduled at 2022-05-19 06:13:45.473591+00:00) buildlogsarchiver stdout | 2022-05-19 06:13:45,476 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 5, 4, 6, 13, 45, 475353), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-19 06:13:45,488 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-19 06:13:45,488 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-19 06:13:45,488 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-19 06:14:15 UTC)" executed successfully namespacegcworker stdout | 2022-05-19 06:13:46,492 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-19 06:13:46,492 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:14:16.679426+00:00 (in 30.187128 seconds) namespacegcworker stdout | 2022-05-19 06:13:46,492 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:14:46 UTC)" (scheduled at 2022-05-19 06:13:46.491684+00:00) namespacegcworker stdout | 2022-05-19 06:13:46,495 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-19 06:13:46,496 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 13, 46, 495308), True, datetime.datetime(2022, 5, 19, 6, 13, 46, 495308), 0, 'namespacegc/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-19 06:13:46,519 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-6d5445778f-pqw25', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} namespacegcworker stdout | 2022-05-19 06:13:46,523 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-19 06:13:46,524 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-19 06:13:46,524 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-19 06:14:46 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:13:46,822 [235] [DEBUG] [app] Starting request: urn:request:3f3c4286-d91e-4ba6-b3a5-ee27c984574a (/api/v1/repository) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:46,822 [235] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:46,823 [235] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:46,823 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:46,834 [235] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:46,834 [235] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:46,834 [235] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:46,837 [235] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:46,837 [235] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:46,837 [235] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:46,837 [235] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:46,837 [235] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2022-05-19 06:13:46,840 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-web stdout | 2022-05-19 06:13:46,845 [235] [DEBUG] [auth.permissions] Organization team added permission: namespacewide(type='organization', namespace='build_test', role='admin') gunicorn-web stdout | 2022-05-19 06:13:46,845 [235] [DEBUG] [auth.permissions] Organization team added repo permission: namespacewide(type='organizationrepo', namespace='build_test', role='admin') gunicorn-web stdout | 2022-05-19 06:13:46,845 [235] [DEBUG] [auth.permissions] Team added permission: teamwideneed(type='orgteam', orgname='build_test', teamname='owners', role='admin') gunicorn-web stdout | 2022-05-19 06:13:46,848 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:46,853 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['build_test', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:46,857 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:46,860 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1"', []) gunicorn-web stdout | 2022-05-19 06:13:46,862 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-web stdout | 2022-05-19 06:13:46,865 [235] [DEBUG] [peewee] ('INSERT INTO "repository" ("namespace_user_id", "name", "visibility_id", "description", "badge_token", "kind_id", "trust_enabled", "state") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "repository"."id"', [2, 'github', 2, '', '25144593-3ac0-499e-980e-cf3e8d44ba5a', 1, False, 0]) gunicorn-web stdout | 2022-05-19 06:13:46,869 [235] [DEBUG] [peewee] ('INSERT INTO "repositoryactioncount" ("repository_id", "count", "date") VALUES (%s, %s, %s) RETURNING "repositoryactioncount"."id"', [1, 0, datetime.date(2022, 5, 18)]) gunicorn-web stdout | 2022-05-19 06:13:46,872 [235] [DEBUG] [peewee] ('INSERT INTO "repositorysearchscore" ("repository_id", "score") VALUES (%s, %s) RETURNING "repositorysearchscore"."id"', [1, 0]) gunicorn-web stdout | 2022-05-19 06:13:46,873 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['admin', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:46,875 [235] [DEBUG] [peewee] ('INSERT INTO "repositorypermission" ("user_id", "repository_id", "role_id") VALUES (%s, %s, %s) RETURNING "repositorypermission"."id"', [1, 1, 1]) gunicorn-web stdout | 2022-05-19 06:13:46,881 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" WHERE ((("t1"."org_id" = %s) AND (("t1"."activating_user_id" = %s) OR ("t1"."activating_user_id" IS %s))) AND ("t1"."delegate_user_id" IS %s))', [2, 1, None, None]) gunicorn-web stdout | 2022-05-19 06:13:46,883 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."org_id", "t1"."uuid", "t1"."activating_user_id", "t1"."delegate_user_id", "t1"."delegate_team_id", "t1"."role_id" FROM "permissionprototype" AS "t1" WHERE ((("t1"."org_id" = %s) AND (("t1"."activating_user_id" = %s) OR ("t1"."activating_user_id" IS %s))) AND ("t1"."delegate_team_id" IS %s))', [2, 1, None, None]) gunicorn-web stdout | 2022-05-19 06:13:46,886 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:46,890 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['build_test', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:46,892 [235] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "logentrykind" AS "t1"', []) gunicorn-web stdout | 2022-05-19 06:13:46,896 [235] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [14, 2, 1, 1, datetime.datetime(2022, 5, 19, 6, 13, 46, 889318), '10.129.2.1', '{"repo": "github", "namespace": "build_test"}']) gunicorn-web stdout | 2022-05-19 06:13:46,902 [235] [DEBUG] [app] Ending request: urn:request:3f3c4286-d91e-4ba6-b3a5-ee27c984574a (/api/v1/repository) {'endpoint': 'api.repositorylist', 'request_id': 'urn:request:3f3c4286-d91e-4ba6-b3a5-ee27c984574a', 'remote_addr': '10.129.2.1', 'http_method': 'POST', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/repository', 'path': '/api/v1/repository', 'parameters': {}, 'json_body': {'namespace': 'build_test', 'repository': 'github', 'visibility': 'private', 'description': '', 'repo_kind': 'image'}, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:46,904 [235] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:13:46,904 [235] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:46 +0000] "POST /api/v1/repository HTTP/1.0" 201 63 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/new/?namespace=build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:46 +0000] "POST /api/v1/repository HTTP/1.1" 201 63 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/new/?namespace=build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.083 1875 0.083) gunicorn-web stdout | 2022-05-19 06:13:47,325 [233] [DEBUG] [app] Starting request: urn:request:258b9c36-81d7-4a11-8801-b218481b00f3 (/api/v1/repository/build_test/github/tag/) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:47,326 [233] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:47,326 [233] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:47,327 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:47,336 [231] [DEBUG] [app] Starting request: urn:request:0df64cd4-d25e-4e77-bb80-56904c41bd0b (/api/v1/repository/build_test/github) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:47,337 [231] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:47,337 [231] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:47,338 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:47,343 [233] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:47,343 [233] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:47,343 [233] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:47,343 [233] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2022-05-19 06:13:47,343 [233] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:47,343 [233] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:47,343 [233] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:47,343 [233] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:47,343 [233] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2022-05-19 06:13:47,347 [233] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2022-05-19 06:13:47,347 [231] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:47,347 [231] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:47,348 [231] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:47,348 [231] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2022-05-19 06:13:47,348 [231] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:47,348 [231] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:47,348 [231] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:47,348 [231] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:47,348 [231] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2022-05-19 06:13:47,352 [231] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2022-05-19 06:13:47,356 [233] [DEBUG] [auth.permissions] User added permission: resource(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2022-05-19 06:13:47,358 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:47,362 [231] [DEBUG] [auth.permissions] User added permission: resource(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2022-05-19 06:13:47,363 [231] [DEBUG] [endpoints.api.repository] Get repo: build_test/github gunicorn-web stdout | 2022-05-19 06:13:47,363 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:47,365 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:47,366 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['public', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:47,369 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."repository_id", "t1"."created" FROM "star" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."user_id" = %s)) LIMIT %s OFFSET %s', [1, 1, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:47,370 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."repository_id", "t1"."manifest_id", "t1"."lifetime_start_ms", "t1"."lifetime_end_ms", "t1"."hidden", "t1"."reversion", "t1"."tag_kind_id", "t1"."linked_tag_id", "t2"."id", "t2"."digest", "t2"."media_type_id", "t2"."layers_compressed_size", "t2"."config_media_type" FROM "tag" AS "t1" INNER JOIN "manifest" AS "t2" ON ("t1"."manifest_id" = "t2"."id") WHERE ((("t1"."repository_id" = %s) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) ORDER BY "t1"."lifetime_start_ms" DESC, "t1"."name" LIMIT %s OFFSET %s', [1, None, 1652940827369, False, 101, 0]) gunicorn-web stdout | 2022-05-19 06:13:47,372 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['public', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:47,375 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-web stdout | 2022-05-19 06:13:47,375 [233] [DEBUG] [app] Ending request: urn:request:258b9c36-81d7-4a11-8801-b218481b00f3 (/api/v1/repository/build_test/github/tag/) {'endpoint': 'api.listrepositorytags', 'request_id': 'urn:request:258b9c36-81d7-4a11-8801-b218481b00f3', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/repository/build_test/github/tag/?limit=100&page=1&onlyActiveTags=true', 'path': '/api/v1/repository/build_test/github/tag/', 'parameters': {'limit': '100', 'page': '1', 'onlyActiveTags': 'true'}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:47,376 [233] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:13:47,377 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:47 +0000] "GET /api/v1/repository/build_test/github/tag/?limit=100&page=1&onlyActiveTags=true HTTP/1.0" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github?namespace=build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" gunicorn-web stdout | 2022-05-19 06:13:47,378 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."count", "t1"."date" FROM "repositoryactioncount" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."date" >= %s))', [1, datetime.date(2022, 2, 16)]) nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:47 +0000] "GET /api/v1/repository/build_test/github/tag/?limit=100&page=1&onlyActiveTags=true HTTP/1.1" 200 49 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github?namespace=build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.052 1692 0.053) gunicorn-web stdout | 2022-05-19 06:13:47,384 [231] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:47,384 [231] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:47,388 [231] [DEBUG] [app] Ending request: urn:request:0df64cd4-d25e-4e77-bb80-56904c41bd0b (/api/v1/repository/build_test/github) {'endpoint': 'api.repository', 'request_id': 'urn:request:0df64cd4-d25e-4e77-bb80-56904c41bd0b', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/repository/build_test/github?includeStats=true&includeTags=false', 'path': '/api/v1/repository/build_test/github', 'parameters': {'includeStats': 'true', 'includeTags': 'false'}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:47,389 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-19 06:13:47,390 [231] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:47 +0000] "GET /api/v1/repository/build_test/github?includeStats=true&includeTags=false HTTP/1.0" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github?namespace=build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:47 +0000] "GET /api/v1/repository/build_test/github?includeStats=true&includeTags=false HTTP/1.1" 200 3626 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github?namespace=build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.056 1686 0.057) notificationworker stdout | 2022-05-19 06:13:48,405 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:13:48,405 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:49.297880+00:00 (in 0.892161 seconds) notificationworker stdout | 2022-05-19 06:13:48,406 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:14:48 UTC)" (scheduled at 2022-05-19 06:13:48.405016+00:00) notificationworker stdout | 2022-05-19 06:13:48,406 [73] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-19 06:13:48,406 [73] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-19 06:14:48 UTC)" executed successfully gunicorn-web stdout | 2022-05-19 06:13:48,868 [233] [DEBUG] [app] Starting request: urn:request:0d6a9393-205f-4f7f-9055-46808e74eae9 (/repository/build_test/github/status) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:48,869 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:48,883 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:48,887 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "visibility" AS "t3" ON ("t1"."visibility_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t3"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 'public', 3, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:48,894 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s OFFSET %s', ['github', 'build_test', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:48,904 [233] [DEBUG] [app] Ending request: urn:request:0d6a9393-205f-4f7f-9055-46808e74eae9 (/repository/build_test/github/status) {'endpoint': 'web.build_status_badge', 'request_id': 'urn:request:0d6a9393-205f-4f7f-9055-46808e74eae9', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github/status?token=25144593-3ac0-499e-980e-cf3e8d44ba5a', 'path': '/repository/build_test/github/status', 'parameters': {'token': '25144593-3ac0-499e-980e-cf3e8d44ba5a'}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:48,905 [233] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:48 +0000] "GET /repository/build_test/github/status?token=25144593-3ac0-499e-980e-cf3e8d44ba5a HTTP/1.1" 200 399 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github?namespace=build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.039 1613 0.040) gunicorn-web stdout | 2022-05-19 06:13:48,906 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:48 +0000] "GET /repository/build_test/github/status?token=25144593-3ac0-499e-980e-cf3e8d44ba5a HTTP/1.0" 200 740 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github?namespace=build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" gunicorn-web stdout | 2022-05-19 06:13:49,178 [233] [DEBUG] [app] Starting request: urn:request:56ede4cb-f6f2-45ab-a24f-a33f0e7a71fd (/api/v1/repository/build_test/github/build/) {'X-Forwarded-For': '10.72.13.64, 10.129.2.1'} gunicorn-web stdout | 2022-05-19 06:13:49,179 [233] [DEBUG] [app] User loader loading deferred user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:49,179 [233] [DEBUG] [auth.cookie] Loading user from cookie: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:49,180 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['ea22316a-d184-4d51-8b14-7f882e8f8e28', False, 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:49,189 [233] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-19 06:13:49,190 [233] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-19 06:13:49,190 [233] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:49,190 [233] [DEBUG] [endpoints.api] Checking permission for repo: build_test/github gunicorn-web stdout | 2022-05-19 06:13:49,190 [233] [DEBUG] [auth.permissions] Loading user permissions after deferring for: ea22316a-d184-4d51-8b14-7f882e8f8e28 gunicorn-web stdout | 2022-05-19 06:13:49,190 [233] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:49,190 [233] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:49,191 [233] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='whuquay', role='admin') gunicorn-web stdout | 2022-05-19 06:13:49,191 [233] [DEBUG] [auth.permissions] Adding superuser to user: whuquay gunicorn-web stdout | 2022-05-19 06:13:49,194 [233] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', ['github', 'build_test', 1, 'github', 'build_test', 1]) gunicorn-web stdout | 2022-05-19 06:13:49,203 [233] [DEBUG] [auth.permissions] User added permission: resource(type='repository', namespace='build_test', name='github', role='admin') gunicorn-web stdout | 2022-05-19 06:13:49,205 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['build_test', 'github', 3, 'application', 1, 0]) gunicorn-web stdout | 2022-05-19 06:13:49,211 [233] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."access_token_id", "t1"."resource_key", "t1"."job_config", "t1"."phase", "t1"."started", "t1"."display_name", "t1"."trigger_id", "t1"."pull_robot_id", "t1"."logs_archived", "t1"."queue_id", "t2"."id", "t2"."uuid", "t2"."service_id", "t2"."repository_id", "t2"."connected_user_id", "t2"."secure_auth_token", "t2"."secure_private_key", "t2"."fully_migrated", "t2"."config", "t2"."write_token_id", "t2"."pull_robot_id", "t2"."enabled", "t2"."disabled_reason_id", "t2"."disabled_datetime", "t2"."successive_failure_count", "t2"."successive_internal_error_count", "t3"."id", "t3"."name", "t4"."id", "t4"."namespace_user_id", "t4"."name", "t4"."visibility_id", "t4"."description", "t4"."badge_token", "t4"."kind_id", "t4"."trust_enabled", "t4"."state", "t5"."id", "t5"."uuid", "t5"."username", "t5"."password_hash", "t5"."email", "t5"."verified", "t5"."stripe_id", "t5"."organization", "t5"."robot", "t5"."invoice_email", "t5"."invalid_login_attempts", "t5"."last_invalid_login", "t5"."removed_tag_expiration_s", "t5"."enabled", "t5"."invoice_email_address", "t5"."given_name", "t5"."family_name", "t5"."company", "t5"."location", "t5"."maximum_queued_builds_count", "t5"."creation_date", "t5"."last_accessed", "t6"."id", "t6"."uuid", "t6"."username", "t6"."password_hash", "t6"."email", "t6"."verified", "t6"."stripe_id", "t6"."organization", "t6"."robot", "t6"."invoice_email", "t6"."invalid_login_attempts", "t6"."last_invalid_login", "t6"."removed_tag_expiration_s", "t6"."enabled", "t6"."invoice_email_address", "t6"."given_name", "t6"."family_name", "t6"."company", "t6"."location", "t6"."maximum_queued_builds_count", "t6"."creation_date", "t6"."last_accessed" FROM "repositorybuild" AS "t1" INNER JOIN "repository" AS "t4" ON ("t1"."repository_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t4"."namespace_user_id" = "t5"."id") LEFT OUTER JOIN "user" AS "t6" ON ("t1"."pull_robot_id" = "t6"."id") LEFT OUTER JOIN "repositorybuildtrigger" AS "t2" ON ("t1"."trigger_id" = "t2"."id") LEFT OUTER JOIN "buildtriggerservice" AS "t3" ON ("t2"."service_id" = "t3"."id") WHERE (("t4"."name" = %s) AND ("t5"."username" = %s)) ORDER BY "t1"."started" DESC LIMIT %s', ['github', 'build_test', 3]) gunicorn-web stdout | 2022-05-19 06:13:49,217 [233] [DEBUG] [app] Ending request: urn:request:56ede4cb-f6f2-45ab-a24f-a33f0e7a71fd (/api/v1/repository/build_test/github/build/) {'endpoint': 'api.repositorybuildlist', 'request_id': 'urn:request:56ede4cb-f6f2-45ab-a24f-a33f0e7a71fd', 'remote_addr': '10.129.2.1', 'http_method': 'GET', 'original_url': 'https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/api/v1/repository/build_test/github/build/?limit=3', 'path': '/api/v1/repository/build_test/github/build/', 'parameters': {'limit': '3'}, 'json_body': None, 'confsha': 'cf0fa2b0', 'user-agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36'} gunicorn-web stdout | 2022-05-19 06:13:49,218 [233] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.129.2.1 () - - [19/May/2022:06:13:49 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/1.1" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github?namespace=build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" (0.042 1665 0.042) gunicorn-web stdout | 2022-05-19 06:13:49,219 [233] [INFO] [gunicorn.access] 10.129.2.1 - - [19/May/2022:06:13:49 +0000] "GET /api/v1/repository/build_test/github/build/?limit=3 HTTP/1.0" 200 15 "https://quayregistry-quay-quay-enterprise.apps.whu49osp12.0519-xot.qe.rhcloud.com/repository/build_test/github?namespace=build_test" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" notificationworker stdout | 2022-05-19 06:13:49,298 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-19 06:13:49,298 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-19 06:13:59.297880+00:00 (in 9.999182 seconds) notificationworker stdout | 2022-05-19 06:13:49,299 [73] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-19 06:13:59 UTC)" (scheduled at 2022-05-19 06:13:49.297880+00:00) notificationworker stdout | 2022-05-19 06:13:49,299 [73] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-19 06:13:49,301 [73] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 19, 6, 13, 49, 299754), True, datetime.datetime(2022, 5, 19, 6, 13, 49, 299754), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-19 06:13:49,317 [73] [DEBUG