-
Bug
-
Resolution: Won't Do
-
Major
-
None
-
None
-
False
-
False
-
Quay Enterprise
-
Logs from the pod:
ibazulic@cyberdyne:~$ oc logs quay-operator.v3.5.7-bffbcd6cd-p88q8
I1012 11:04:32.078185 1 request.go:655] Throttling request took 1.039889923s, request: GET:https://10.217.4.1:443/apis/apiserver.openshift.io/v1?timeout=32s
2021-10-12T11:04:33.834Z INFO controller-runtime.metrics metrics server is starting to listen {"addr": ":8080"}
2021-10-12T11:04:33.834Z INFO controller-runtime.injectors-warning Injectors are deprecated, and will be removed in v0.10.x
2021-10-12T11:04:33.834Z INFO controller-runtime.injectors-warning Injectors are deprecated, and will be removed in v0.10.x
2021-10-12T11:04:33.834Z INFO controller-runtime.injectors-warning Injectors are deprecated, and will be removed in v0.10.x
2021-10-12T11:04:33.834Z INFO controller-runtime.injectors-warning Injectors are deprecated, and will be removed in v0.10.x
2021-10-12T11:04:33.834Z INFO setup starting server on port 7071
2021-10-12T11:04:33.834Z INFO setup starting manager
2021-10-12T11:04:33.835Z INFO controller-runtime.manager starting metrics server {"path": "/metrics"}
2021-10-12T11:04:33.835Z INFO controller-runtime.manager.controller.quayecosystem Starting EventSource {"reconciler group": "redhatcop.redhat.io", "reconciler kind": "QuayEcosystem", "source": "kind source: /, Kind="}
2021-10-12T11:04:33.835Z INFO controller-runtime.manager.controller.quayregistry Starting EventSource {"reconciler group": "quay.redhat.com", "reconciler kind": "QuayRegistry", "source": "kind source: /, Kind="}
2021-10-12T11:04:33.936Z INFO controller-runtime.manager.controller.quayecosystem Starting Controller {"reconciler group": "redhatcop.redhat.io", "reconciler kind": "QuayEcosystem"}
2021-10-12T11:04:33.936Z INFO controller-runtime.manager.controller.quayecosystem Starting workers {"reconciler group": "redhatcop.redhat.io", "reconciler kind": "QuayEcosystem", "worker count": 1}
2021-10-12T11:04:33.936Z INFO controller-runtime.manager.controller.quayregistry Starting Controller {"reconciler group": "quay.redhat.com", "reconciler kind": "QuayRegistry"}
2021-10-12T11:04:33.936Z INFO controller-runtime.manager.controller.quayregistry Starting workers {"reconciler group": "quay.redhat.com", "reconciler kind": "QuayRegistry", "worker count": 1}
2021-10-12T11:04:33.936Z INFO controllers.QuayRegistry begin reconcile {"quayregistry": "quay-enterprise/example-registry"}
2021-10-12T11:04:34.337Z INFO controllers.QuayRegistry successfully retrieved referenced `configBundleSecret` {"quayregistry": "quay-enterprise/example-registry", "configBundleSecret": "quay-config-bundle", "resourceVersion": "48316"}
2021-10-12T11:04:34.350Z INFO controllers.QuayRegistry cluster supports `Routes` API
2021-10-12T11:04:34.951Z INFO controllers.QuayRegistry Detected cluster hostname apps-crc.testing
2021-10-12T11:04:34.951Z INFO controllers.QuayRegistry detected router canonical hostname: apps-crc.testing
2021-10-12T11:04:38.131Z INFO controllers.QuayRegistry cluster does not support `ObjectBucketClaim` API
2021-10-12T11:04:38.231Z INFO controllers.QuayRegistry cluster supports `ServiceMonitor` API
2021-10-12T11:04:38.332Z INFO controllers.QuayRegistry cluster supports `PrometheusRules` API
2021-10-12T11:04:38.433Z INFO controllers.QuayRegistry openshift-config-managed found
2021-10-12T11:04:39.440Z INFO controllers.QuayRegistry inflating QuayRegistry into Kubernetes objects using Kustomize {"quayregistry": "quay-enterprise/example-registry"}
2021-10-12T11:04:39.440Z INFO controllers.QuayRegistry `DATABASE_SECRET_KEY` not found in user-provided config, generating new one {"quayregistry": "quay-enterprise/example-registry"}
2021-10-12T11:04:39.440Z INFO controllers.QuayRegistry `SECRET_KEY` not found in user-provided config, generating new one {"quayregistry": "quay-enterprise/example-registry"}
E1012 11:04:39.440444 1 runtime.go:78] Observed a panic: "assignment to entry in nil map" (assignment to entry in nil map)
goroutine 436 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic(0x27f1a80, 0x2d477e0)
/workspace/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:74 +0x95
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
/workspace/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:48 +0x89
panic(0x27f1a80, 0x2d477e0)
/opt/rh/go-toolset-1.15/root/usr/lib/go-toolset-1.15-golang/src/runtime/panic.go:969 +0x1b9
github.com/quay/quay-operator/pkg/kustomize.Inflate(0xc000c3a160, 0xc0067e9dc0, 0xc000810000, 0x2dd17e0, 0xc000654150, 0x0, 0x0, 0xd, 0xc0069780e0, 0x6c)
/workspace/pkg/kustomize/kustomize.go:443 +0x171
github.com/quay/quay-operator/controllers/quay.(*QuayRegistryReconciler).Reconcile(0xc00061c000, 0x2dc4b20, 0xc000566150, 0xc00012d5d0, 0xf, 0xc00012d5b0, 0x10, 0xc000566150, 0xdd63ff, 0xc00003c000, ...)
/workspace/controllers/quay/quayregistry_controller.go:269 +0x22e7
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000021180, 0x2dc4a60, 0xc000894000, 0x2878360, 0xc00000eda0)
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:297 +0x317
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000021180, 0x2dc4a60, 0xc000894000, 0xc00062be00)
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:252 +0x205
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2(0x2dc4a60, 0xc000894000)
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:215 +0x4a
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1()
/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:185 +0x37
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00062bf50)
/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc006107f50, 0x2d72d00, 0xc0005660c0, 0xc000894001, 0xc000896120)
/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156 +0xad
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00062bf50, 0x3b9aca00, 0x0, 0x1, 0xc000896120)
/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext(0x2dc4a60, 0xc000894000, 0xc0006540f0, 0x3b9aca00, 0x0, 0x1)
/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:185 +0xa6
k8s.io/apimachinery/pkg/util/wait.UntilWithContext(0x2dc4a60, 0xc000894000, 0xc0006540f0, 0x3b9aca00)
/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:99 +0x57
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:212 +0x45e
panic: assignment to entry in nil map [recovered]
panic: assignment to entry in nil map
goroutine 436 [running]:
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
/workspace/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:55 +0x10c
panic(0x27f1a80, 0x2d477e0)
/opt/rh/go-toolset-1.15/root/usr/lib/go-toolset-1.15-golang/src/runtime/panic.go:969 +0x1b9
github.com/quay/quay-operator/pkg/kustomize.Inflate(0xc000c3a160, 0xc0067e9dc0, 0xc000810000, 0x2dd17e0, 0xc000654150, 0x0, 0x0, 0xd, 0xc0069780e0, 0x6c)
/workspace/pkg/kustomize/kustomize.go:443 +0x171
github.com/quay/quay-operator/controllers/quay.(*QuayRegistryReconciler).Reconcile(0xc00061c000, 0x2dc4b20, 0xc000566150, 0xc00012d5d0, 0xf, 0xc00012d5b0, 0x10, 0xc000566150, 0xdd63ff, 0xc00003c000, ...)
/workspace/controllers/quay/quayregistry_controller.go:269 +0x22e7
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000021180, 0x2dc4a60, 0xc000894000, 0x2878360, 0xc00000eda0)
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:297 +0x317
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000021180, 0x2dc4a60, 0xc000894000, 0xc00062be00)
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:252 +0x205
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2(0x2dc4a60, 0xc000894000)
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:215 +0x4a
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1()
/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:185 +0x37
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00062bf50)
/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc006107f50, 0x2d72d00, 0xc0005660c0, 0xc000894001, 0xc000896120)
/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156 +0xad
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00062bf50, 0x3b9aca00, 0x0, 0x1, 0xc000896120)
/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext(0x2dc4a60, 0xc000894000, 0xc0006540f0, 0x3b9aca00, 0x0, 0x1)
/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:185 +0xa6
k8s.io/apimachinery/pkg/util/wait.UntilWithContext(0x2dc4a60, 0xc000894000, 0xc0006540f0, 0x3b9aca00)
/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:99 +0x57
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:212 +0x45e
I've used the most basic config.yaml file that has nothing except my storage definition:
DISTRIBUTED_STORAGE_CONFIG: default: - RadosGWStorage - access_key: minioadmin bucket_name: quay-openshift hostname: 172.24.0.1 is_secure: true port: "9000" secret_key: minioadmin storage_path: /datastorage/registry DISTRIBUTED_STORAGE_DEFAULT_LOCATIONS: - default DISTRIBUTED_STORAGE_PREFERENCE: - default
The storage definition works on Quay standalone and I don't think it's the cause of the problem since the logs show that my config was read and failure is somewhere else.
