CRW 2.12.0.RC-10-11 with operator image https://quay.io/crw/crw-2-rhel8-operator@sha256:458802564043f3047e477045c9cf4e06ef6b9e41bd8de3a05f482250f326ea28 installation failed on operator container creation:
https://main-jenkins-csb-crwqe.apps.ocp4.prod.psi.redhat.com/job/CodereadyWorkspaces/job/e2e/job/basic/job/install-crw-using-crwctl/1530/console
codeready-operator-9b95f58f8-2gjh2.pod.log
I1012 04:11:32.506528 1 request.go:655] Throttling request took 2.340457604s, request: GET:https://172.30.0.1:443/apis/quota.openshift.io/v1?timeout=32s {"level":"error","ts":1634011894.4886575,"logger":"controller","msg":"Reconciler error","reconcilerGroup":"org.eclipse.che","reconcilerKind":"CheCluster","controller":"checluster","name":"codeready-workspaces","namespace":"crw-crwctl","error":"Operation cannot be fulfilled on checlusters.org.eclipse.che \"codeready-workspaces\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/che-operator/vendor/github.com/go-logr/zapr/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/che-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:246\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/che-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:218\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/che-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:197\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/che-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/che-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/che-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/che-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90"}
+ /mnt/hudson_workspace/workspace/CodereadyWorkspaces/e2e/basic/install-crw-using-crwctl/crwctl/bin/crwctl server:deploy --che-operator-cr-patch-yaml=/mnt/hudson_workspace/workspace/CodereadyWorkspaces/e2e/basic/install-crw-using-crwctl/custom-resource-patch.yaml --no-olm-suggested-namespace --catalog-source-name=crw-2-12-osbs-latest --catalog-source-namespace=openshift-marketplace --package-manifest-name=codeready-workspaces --olm-channel=latest --listr-renderer=verbose -n crw-crwctl › Installer type is set to: 'olm' [04:03:27] Verify Kubernetes API [started] › Current Kubernetes context: 'default/api-ocp48-aws-crw-qe-com:6443/admin' [04:03:27] Verify Kubernetes API...OK [title changed] [04:03:28] Verify Kubernetes API...OK (it's OpenShift) [title changed] [04:03:28] Verify Kubernetes API...OK (it's OpenShift) [completed] [04:03:28] Looking for an already existing CodeReady Workspaces instance [started] [04:03:28] Verify if CodeReady Workspaces is deployed into namespace "crw-crwctl" [started] [04:03:28] Verify if CodeReady Workspaces is deployed into namespace "crw-crwctl"...it is not [title changed] [04:03:28] Verify if CodeReady Workspaces is deployed into namespace "crw-crwctl"...it is not [completed] [04:03:28] Looking for an already existing CodeReady Workspaces instance [completed] [04:03:28] Openshift preflight checklist [started] [04:03:28] Verify if oc is installed [started] [04:03:28] Verify if oc is installed...done. [title changed] [04:03:28] Verify if oc is installed...done. [completed] [04:03:28] Verify if openshift is running [started] [04:03:29] Verify if openshift is running...done. [title changed] [04:03:29] Verify if openshift is running...done. [completed] [04:03:29] Check OpenShift version [started] [04:03:30] Check OpenShift version: 4.x [title changed] [04:03:30] Check OpenShift version: 4.x [completed] [04:03:30] Check Kubernetes version [started] [04:03:31] Check Kubernetes version: Unknown. [title changed] [04:03:31] Check Kubernetes version: Unknown. [completed] [04:03:31] Openshift preflight checklist [completed] [04:03:31] Following CodeReady Workspaces logs [started] [04:03:31] Start following Operator logs [started] [04:03:31] Start following Operator logs...done [title changed] [04:03:31] Start following Operator logs...done [completed] [04:03:31] Start following CodeReady Workspaces Server logs [started] [04:03:31] Start following CodeReady Workspaces Server logs...done [title changed] [04:03:31] Start following CodeReady Workspaces Server logs...done [completed] [04:03:31] Start following PostgreSQL logs [started] [04:03:31] Start following PostgreSQL logs...done [title changed] [04:03:31] Start following PostgreSQL logs...done [completed] [04:03:31] Start following Keycloak logs [started] [04:03:31] Start following Keycloak logs...done [title changed] [04:03:31] Start following Keycloak logs...done [completed] [04:03:31] Start following Plug-in Registry logs [started] [04:03:31] Start following Plug-in Registry logs...done [title changed] [04:03:31] Start following Plug-in Registry logs...done [completed] [04:03:31] Start following Devfile Registry logs [started] [04:03:31] Start following Devfile Registry logs...done [title changed] [04:03:31] Start following Devfile Registry logs...done [completed] [04:03:31] Start following CodeReady Workspaces Dashboard logs [started] [04:03:31] Start following CodeReady Workspaces Dashboard logs...done [title changed] [04:03:31] Start following CodeReady Workspaces Dashboard logs...done [completed] [04:03:31] Start following namespace events [started] [04:03:31] Start following namespace events...done [title changed] [04:03:31] Start following namespace events...done [completed] [04:03:31] Following CodeReady Workspaces logs [completed] [04:03:31] Running Olm installaion CodeReady Workspaces [started] [04:03:31] Check if OLM is pre-installed on the platform [started] [04:03:31] Check if OLM is pre-installed on the platform...done. [title changed] [04:03:31] Check if OLM is pre-installed on the platform...done. [completed] [04:03:31] Create Namespace (crw-crwctl) [started] [04:03:32] Create Namespace (crw-crwctl)...Done. [title changed] [04:03:32] Create Namespace (crw-crwctl)...Done. [completed] [04:03:32] Create operator group [started] [04:03:33] Create operator group...created new one. [title changed] [04:03:33] Create operator group...created new one. [completed] [04:03:33] Configure context information [started] [04:03:33] Configure context information...done. [title changed] [04:03:33] Configure context information...done. [completed] [04:03:33] Create operator subscription [started] [04:03:33] Create operator subscription...created new one. [title changed] [04:03:33] Create operator subscription...created new one. [completed] [04:03:33] Wait while subscription is ready [started] [04:03:34] Wait while subscription is ready...done. [title changed] [04:03:34] Wait while subscription is ready...done. [completed] [04:03:34] Wait while operator installed [started] [04:03:37] Wait while operator installed...done. [title changed] [04:03:37] Wait while operator installed...done. [completed] [04:03:37] Prepare CodeReady Workspaces cluster CR [started] [04:03:38] Prepare CodeReady Workspaces cluster CR...Done. [title changed] [04:03:38] Prepare CodeReady Workspaces cluster CR...Done. [completed] [04:03:38] Create the Custom Resource of type checlusters.org.eclipse.che in the namespace crw-crwctl [started] [04:03:39] Create the Custom Resource of type checlusters.org.eclipse.che in the namespace crw-crwctl...done. [title changed] [04:03:39] Create the Custom Resource of type checlusters.org.eclipse.che in the namespace crw-crwctl...done. [completed] [04:03:39] Running Olm installaion CodeReady Workspaces [completed] [04:03:39] Post installation checklist [started] [04:03:39] PostgreSQL pod bootstrap [started] [04:03:39] Scheduling [started] [04:31:25] Scheduling [failed] [04:31:25] Timeout: there are no pods in the namespace: crw-crwctl, selector: app=codeready,component=postgres. Check CodeReady Workspaces logs for details. Consider increasing error recheck timeout with --k8spoderrorrechecktimeout flag. [04:31:25] PostgreSQL pod bootstrap [failed] [04:31:25] Timeout: there are no pods in the namespace: crw-crwctl, selector: app=codeready,component=postgres. Check CodeReady Workspaces logs for details. Consider increasing error recheck timeout with --k8spoderrorrechecktimeout flag. [04:31:25] Post installation checklist [failed] [04:31:25] Timeout: there are no pods in the namespace: crw-crwctl, selector: app=codeready,component=postgres. Check CodeReady Workspaces logs for details. Consider increasing error recheck timeout with --k8spoderrorrechecktimeout flag. Error: Command server:deploy failed. Error log: /home/hudson/.cache/crwctl/error.log.
OCP version: 4.8
Operator: quay.io/repository/crw/crw-2-rhel8-operator:2.12-9
Recent changes: https://github.com/eclipse-che/che-operator/blob/main/main.go#L229
Update: there was CRW from tech-preview channel installed before.
Uninstallation didn't help.
Update 2: following advice from sleshche, CRW has successfully finished installation after removal of all installed CRW 2.12.0 instances from OCP 4.8 cluster:
I1012 16:21:46.761797 1 request.go:655] Throttling request took 1.014377982s, request: GET:https://172.30.0.1:443/apis/k8s.cni.cncf.io/v1?timeout=32s {"level":"error","ts":1634055713.5063508,"logger":"controller","msg":"Reconciler error","reconcilerGroup":"org.eclipse.che","reconcilerKind":"CheCluster","controller":"checluster","name":"codeready-workspaces","namespace":"crw-crwctl","error":"Operation cannot be fulfilled on checlusters.org.eclipse.che \"codeready-workspaces\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/che-operator/vendor/github.com/go-logr/zapr/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/che-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:246\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/che-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:218\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/che-operator/vendor/sigs.k8s... I1012 16:21:56.762026 1 request.go:655] Throttling request took 2.248773151s, request: GET:https://172.30.0.1:443/apis/network.operator.openshift.io/v1?timeout=32s time="2021-10-12T16:22:02Z" level=info msg="Custom resource spec codeready-workspaces updated with openShiftoAuth: true" time="2021-10-12T16:22:04Z" level=info msg="Creating a new object: v1.Secret, name: self-signed-certificate" time="2021-10-12T16:22:04Z" level=info msg="Creating a new object: v1.Secret, name: openshift-api-crt" time="2021-10-12T16:22:04Z" level=info msg="Creating a new object: v1.ConfigMap, name: ca-certs-merged" time="2021-10-12T16:22:04Z" level=info msg="Creating a new object: v1.ServiceAccount, name: che" time="2021-10-12T16:22:04Z" level=info msg="Creating a new object: v1.ClusterRole, name: crw-crwctl-cheworkspaces-clusterrole" time="2021-10-12T16:22:04Z" level=info msg="Creating a new object: v1.ClusterRoleBinding, name: crw-crwctl-cheworkspaces-clusterrole" time="2021-10-12T16:22:04Z" level=info msg="Added finalizer: cheWorkspaces.clusterpermissions.finalizers.che.eclipse.org"
Update 3: installation of CRW 2.11.0 failed as well when there is CRW 2.12.0 instance exists on the same cluster:
I1014 08:41:27.649758 1 request.go:655] Throttling request took 1.59789489s, request: GET:https://172.30.0.1:443/apis/scheduling.k8s.io/v1beta1?timeout=32s {"level":"error","ts":1634200890.3639247,"logger":"controller","msg":"Reconciler error","reconcilerGroup":"org.eclipse.che","reconcilerKind":"CheCluster","controller":"checluster","name":"codeready-workspaces","namespace":"crw-2-11-to-12","error":"Operation cannot be fulfilled on checlusters.org.eclipse.che \"codeready-workspaces\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/che-operator/vendor/github.com/go-logr/zapr/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/che-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:246\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/che-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:218\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/che-operator/vendor/sigs...
- is documented by
-
RHDEVDOCS-3374 No more than 1 instance of Che/CRW per cluster supported
- Closed
-
RHDEVDOCS-3375 Document the fact that multiple CRW instances on the same cluster is not supported
- Closed