Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-5065

Installation of WMCO in different namespace fails

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Blocker
    • 4.13.0
    • 4.12
    • Windows Containers
    • None
    • 3
    • WINC - Sprint 230
    • 1
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Hide
      Previously, a regression within the WMCO's subcomponents was preventing installation in custom namespaces. The operator manifests were not properly setting RBAC, aside from the default install namespace. Now proper RBAC is set and WMCO is back to full support in custom namespaces.
      Show
      Previously, a regression within the WMCO's subcomponents was preventing installation in custom namespaces. The operator manifests were not properly setting RBAC, aside from the default install namespace. Now proper RBAC is set and WMCO is back to full support in custom namespaces.
    • Bug Fix

    Description

      Description of problem:

      When installing WMCO in a different namespace than openshift-windows-machine-config-operator, the WMCO operator starts malfunctioning and it isn't able to add nodes to the cluster. When checking the wmco logs we can observe some issue with the WICD user's policy which points to openshift-windows-machine-config-operator:  User \"system:serviceaccount:winc-namespace-test:windows-instance-config-daemon\" cannot get resource \"configmaps\" in API group \"\" in the namespace \"openshift-windows-machine-config-operator\
      
      Logs:
      
      {"level":"error","ts":"2022-12-20T09:58:15Z","logger":"wc 10.0.128.8","msg":"error running","cmd":"powershell.exe -NonInteractive -ExecutionPolicy Bypass \"C:\\k\\windows-instance-config-daemon.exe bootstrap --desired-version 7.0.0-16f486a --api-server https://api-int.jfrancoa-2012-no.qe.azure.devcluster.openshift.com:6443 --sa-ca C:\\k\\sa-ca.crt --sa-token C:\\k\\sa-token\"","out":"I1220 09:58:14.981110    3432 bootstrap.go:61] bootstrapping Windows instance\nF1220 09:58:15.027372    3432 bootstrap.go:63] configmaps \"windows-services-7.0.0-16f486a\" is forbidden: User \"system:serviceaccount:winc-namespace-test:windows-instance-config-daemon\" cannot get resource \"configmaps\" in API group \"\" in the namespace \"openshift-windows-machine-config-operator\"\n","error":"Process exited with status 1","stacktrace":"github.com/openshift/windows-machine-config-operator/pkg/windows.(*windows).Bootstrap\n\t/remote-source/build/windows-machine-config-operator/pkg/windows/windows.go:484\ngithub.com/openshift/windows-machine-config-operator/pkg/nodeconfig.(*nodeConfig).Configure\n\t/remote-source/build/windows-machine-config-operator/pkg/nodeconfig/nodeconfig.go:153\ngithub.com/openshift/windows-machine-config-operator/controllers.(*instanceReconciler).ensureInstanceIsUpToDate\n\t/remote-source/build/windows-machine-config-operator/controllers/controllers.go:91\ngithub.com/openshift/windows-machine-config-operator/controllers.(*WindowsMachineReconciler).addWorkerNode\n\t/remote-source/build/windows-machine-config-operator/controllers/windowsmachine_controller.go:431\ngithub.com/openshift/windows-machine-config-operator/controllers.(*WindowsMachineReconciler).Reconcile\n\t/remote-source/build/windows-machine-config-operator/controllers/windowsmachine_controller.go:353\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/remote-source/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:121\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:320\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:273\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:234"}
      {"level":"info","ts":"2022-12-20T09:58:15Z","logger":"wc 10.0.128.8","msg":"failed to bootstrap node","command":"C:\\k\\windows-instance-config-daemon.exe bootstrap --desired-version 7.0.0-16f486a --api-server https://api-int.jfrancoa-2012-no.qe.azure.devcluster.openshift.com:6443 --sa-ca C:\\k\\sa-ca.crt --sa-token C:\\k\\sa-token","output":"I1220 09:58:14.981110    3432 bootstrap.go:61] bootstrapping Windows instance\nF1220 09:58:15.027372    3432 bootstrap.go:63] configmaps \"windows-services-7.0.0-16f486a\" is forbidden: User \"system:serviceaccount:winc-namespace-test:windows-instance-config-daemon\" cannot get resource \"configmaps\" in API group \"\" in the namespace \"openshift-windows-machine-config-operator\"\n"}
      {"level":"error","ts":"2022-12-20T09:58:15Z","msg":"Reconciler error","controller"
      
      

      Version-Release number of selected component (if applicable):

      $ oc get clusterversion
      NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
      version   4.12.0-0.nightly-2022-12-20-034740   True        False         40m     Cluster version is 4.12.0-0.nightly-2022-12-20-034740
      
      [jfrancoa@localhost wmco]$ oc get cm -n winc-namespace-test
      NAME                                   DATA   AGE
      kube-root-ca.crt                       1      32m
      openshift-service-ca.crt               1      32m
      windows-machine-config-operator-lock   0      28m
      windows-services-7.0.0-16f486a         2      28m
      
      

      How reproducible:

      Always
      

      Steps to Reproduce:

      1. Deploy an OVN Hybrid overlay cluster
      2. Add a catalogsource (named wmco) that provides with the wmco operator bundle
      3. Create a namespace winc-namespace-test in which wmco will be installed:
      apiVersion: v1
      kind: Namespace
      metadata:
        name: winc-namespace-test
        labels:
          openshift.io/cluster-monitoring: "true"
      4. Create the secret in that namespace:
      oc create secret generic cloud-private-key --from-file=private-key.pem=<private-key-location> -n winc-namespace-test
      5. Add the operator group:
      cat operatorgroup.yaml 
      apiVersion: operators.coreos.com/v1
      kind: OperatorGroup
      metadata:
        annotations:
          olm.providedAPIs: WindowsMachineConfig.v1alpha1.wmc.openshift.io
        name: windows-machine-config-operator
        namespace: winc-namespace-test
      spec:
        targetNamespaces:
        - winc-namespace-test
      5. And finally the subscription:
      $ cat operatorgroup.yaml 
      apiVersion: operators.coreos.com/v1
      kind: OperatorGroup
      metadata:
        annotations:
          olm.providedAPIs: WindowsMachineConfig.v1alpha1.wmc.openshift.io
        name: windows-machine-config-operator
        namespace: winc-namespace-test
      spec:
        targetNamespaces:
        - winc-namespace-test
      [jfrancoa@localhost OCP37086]$ cat subscription.yaml 
      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
        name: windows-machine-config-operator
        namespace: winc-namespace-test
      spec:
        channel: stable
        installPlanApproval: Automatic
        name: windows-machine-config-operator
        source: wmco
        sourceNamespace: openshift-marketplace
      6. Once WMCO gets installed, create a MachineSet to start a Windows worker. The node never gets added to the cluster and the machine stays in provisied:
      $ oc get machine.machine.openshift.io -n openshift-machine-api
      NAME                                          PHASE         TYPE              REGION   ZONE   AGE
      jfrancoa-2012-no-4nrfs-master-0               Running       Standard_D8s_v3   eastus   2      67m
      jfrancoa-2012-no-4nrfs-master-1               Running       Standard_D8s_v3   eastus   3      67m
      jfrancoa-2012-no-4nrfs-master-2               Running       Standard_D8s_v3   eastus   1      67m
      jfrancoa-2012-no-4nrfs-worker-eastus1-qtblt   Running       Standard_D4s_v3   eastus   1      62m
      jfrancoa-2012-no-4nrfs-worker-eastus2-8wdvg   Running       Standard_D4s_v3   eastus   2      62m
      jfrancoa-2012-no-4nrfs-worker-eastus3-n6s8h   Running       Standard_D4s_v3   eastus   3      62m
      windows-5spgp                                 Provisioned   Standard_D2s_v3   eastus          31m
      windows-7dxgv                                 Provisioned   Standard_D2s_v3   eastus          31m
      
      

      Actual results:

      WMCO is not able to add Windows nodes to the cluster
      

      Expected results:

      WMCO works as usual and it allows adding new nodes to the cluster
      

      Additional info:

      
      

      Attachments

        Issue Links

          Activity

            People

              mohashai Mohammad Shaikh
              rhn-engineering-jfrancoa Jose Luis Franco Arza (Inactive)
              Aharon Rasouli Aharon Rasouli
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: