Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-14181

Instance configurations fails on Windows Server 2019 without the container feature

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Critical Critical
    • 4.13.z
    • 4.14.0
    • Windows Containers
    • None
    • No
    • 0
    • WINC - Sprint 237
    • 1
    • False
    • Hide

      None

      Show
      None

      This is a clone of issue OCPBUGS-13244. The following is the description of the original issue:

      Description of problem

      WMCO fails to configure Windows Server 2019 instances on Azure that do not have the Containers feature pre-installed with the following error:

      2023-05-05T18:11:02Z	ERROR	wc 10.0.128.7	error running	{"cmd": "powershell.exe -NonInteractive -ExecutionPolicy Bypass \"Install-WindowsFeature -Name Containers\"", "out": "Install-WindowsFeature : Win32 internal error \"Access is denied\" 0x5 occurred while reading the console output buffer. \r\nContact Microsoft Customer Support Services.\r\nAt line:1 char:1\r\n+ Install-WindowsFeature -Name Containers\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n    + CategoryInfo          : ReadError: (:) [Install-WindowsFeature], HostException\r\n    + FullyQualifiedErrorId : ReadConsoleOutput,Microsoft.Windows.ServerManager.Commands.AddWindowsFeatureCommand\r\n \r\n", "error": "Process exited with status 1"}
      github.com/openshift/windows-machine-config-operator/pkg/windows.(*windows).Run
      	/build/windows-machine-config-operator/pkg/windows/windows.go:373
      github.com/openshift/windows-machine-config-operator/pkg/windows.(*windows).enableContainersWindowsFeature
      	/build/windows-machine-config-operator/pkg/windows/windows.go:893
      github.com/openshift/windows-machine-config-operator/pkg/windows.(*windows).ensureHostNameAndContainersFeature
      	/build/windows-machine-config-operator/pkg/windows/windows.go:530
      github.com/openshift/windows-machine-config-operator/pkg/windows.(*windows).Bootstrap
      	/build/windows-machine-config-operator/pkg/windows/windows.go:430
      github.com/openshift/windows-machine-config-operator/pkg/nodeconfig.(*nodeConfig).Configure
      	/build/windows-machine-config-operator/pkg/nodeconfig/nodeconfig.go:159
      github.com/openshift/windows-machine-config-operator/controllers.(*instanceReconciler).ensureInstanceIsUpToDate
      	/build/windows-machine-config-operator/controllers/controllers.go:84
      github.com/openshift/windows-machine-config-operator/controllers.(*WindowsMachineReconciler).configureMachine
      	/build/windows-machine-config-operator/controllers/windowsmachine_controller.go:433
      github.com/openshift/windows-machine-config-operator/controllers.(*WindowsMachineReconciler).Reconcile
      	/build/windows-machine-config-operator/controllers/windowsmachine_controller.go:354
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
      	/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:122
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
      	/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:323
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
      	/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:274
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
      	/build/windows-machine-config-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:235
      
      How reproducible:{code:none}
      Always
      

      Steps to Reproduce:

      1. Bring up a OCP 4.14 cluster on Azure
      2. Create a MachineSet with sku: 2019-datacenter-smalldisk
      3. Launch WMCO from source
      

      Actual results:

      Node fails to get configured
      

      Expected results:

      Node should be configured
      

      Additional info

      The issue here is that Install-WindowsFeature -Name Containers results in a progress bar being displayed and causes the Access is denied error. However I noticed that if you manually installed the containers feature while WMCO was retrying the configuration, the install still does not go through and times out waiting for a hybrid-ovn annotation. In the hybrid-overlay logs I saw:

      F0508 19:28:55.018792    5108 hybrid-overlay-node.go:53] this version of Windows does not support HostRoute policies; network communication between this node and its pods will not work. HostRoute
      policies are available as a KB update for Windows Server 2019 version 1809 and out of the box in Windows Server 2019 version 1903.
      

      Interestingly 2019-datacenter-smalldisk and 2019-datacenter-with-containers-smalldisk have identical patch and hotfixes

      PS C:\Users\capi> [System.Environment]::OSVersion.Version
      
      Major  Minor  Build  Revision
      -----  -----  -----  --------
      10     0      17763  0
      
      PS C:\Users\capi> Get-HotFix
      
      Source        Description      HotFixID      InstalledBy          InstalledOn               
      ------        -----------      --------      -----------          -----------
      win2-mjrss    Update           KB5022511                          4/5/2023 12:00:00 AM
      win2-mjrss    Security Update  KB5025229                          4/5/2023 12:00:00 AM
      win2-mjrss    Security Update  KB5023789                          4/5/2023 12:00:00 AM
      
      

      Note

      It is unclear why the progress bar issue does not show up with 2022 images which do not have the Container feature installed.

              paravindh Aravindh Puthiyaparambil (Inactive)
              openshift-crt-jira-prow OpenShift Prow Bot
              Aharon Rasouli Aharon Rasouli
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: