Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-54255

Windows nodes in disconnected cluster are in NotReady,SchedulingDisabled

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Major Major
    • None
    • 4.18
    • Windows Containers
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      Using Konflux mirrored image registry.redhat.io/openshift4-wincw/windows-machine-config-rhel9-operator@sha256:8b254794428e35bf5c22dab35e25c3abcecb0cd1aed4af8d24d23c5427d73f29
      Disconnected cluster created, however Windows nodes stuck in: 
      winworker-tn8fq                      NotReady,SchedulingDisabled
      
      

       

      Version-Release number of selected component (if applicable):

      "version":"10.18.0-2862b89"    

      How reproducible:

          100% in our CI run

      Steps to Reproduce:

          1. create a disconnected cluster mirror Konflux images with idms for disconnected cluster
          2. create a catalogsource, subscription, operator group
          3. Create Windows machineset      

      Actual results:

          Machine created properly, Windows nodes stuck in NotReady,SchedulingDisabled

      Expected results:

          Windows nodes in a Ready status

      Additional info:

          Noticed in WICD logs:
      E0326 12:55:40.373528     968 controller.go:316] "Reconciler error" err=<
              Failed to read certificate file C:\k\ca-bundle.crt: Failed to parse certificate: x509: malformed certificate
       > controller="node" controllerGroup="" controllerKind="Node" Node="winworker-tn8fq" namespace="" name="winworker-tn8fq" reconcileID="62b263d0-31e3-43e2-a4ca-6a44c5638f13"
      E0326 12:56:00.855549     968 controller.go:316] "Reconciler error" err=<
              Failed to read certificate file C:\k\ca-bundle.crt: Failed to parse certificate: x509: malformed certificate
       > controller="node" controllerGroup="" controllerKind="Node" Node="winworker-tn8fq" namespace="" name="winworker-tn8fq" reconcileID="bc67ef11-142b-4014-ad9a-b75cc82d3694"
      from kubelet log:
      E0326 12:53:13.173898    5624 kubelet.go:2916] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized"
      IDMS:
      oc get idms -oyaml
      apiVersion: v1
      items:
      - apiVersion: config.openshift.io/v1
        kind: ImageDigestMirrorSet
        metadata:
          annotations:
            kubectl.kubernetes.io/last-applied-configuration: |
              {"apiVersion":"config.openshift.io/v1","kind":"ImageDigestMirrorSet","metadata":{"annotations":{},"name":"digested-wmco-mirrors"},"spec":{"imageDigestMirrors":[{"mirrors":["rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-operator-index"],"source":"registry-proxy.engineering.redhat.com/rh-osbs/iib"}]}}
          creationTimestamp: "2025-03-26T09:23:57Z"
          generation: 1
          name: digested-wmco-mirrors
          resourceVersion: "48212"
          uid: 43cb273f-79f1-425d-93cf-4631127a4ddb
        spec:
          imageDigestMirrors:
          - mirrors:
            - rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-operator-index
            source: registry-proxy.engineering.redhat.com/rh-osbs/iib
      - apiVersion: config.openshift.io/v1
        kind: ImageDigestMirrorSet
        metadata:
          annotations:
            kubectl.kubernetes.io/last-applied-configuration: |
              {"apiVersion":"config.openshift.io/v1","kind":"ImageDigestMirrorSet","metadata":{"annotations":{},"name":"image-digest-mirror"},"spec":{"imageDigestMirrors":[{"mirrors":["rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-operator-index"],"source":"registry-proxy.engineering.redhat.com/rh-osbs/iib"},{"mirrors":["rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-operator-index","rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/rh-osbs/iib"],"source":"registry-proxy.engineering.redhat.com/rh-osbs/iib"},{"mirrors":["rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-operator-bundle"],"source":"registry-proxy.engineering.redhat.com/openshift4-wincw/windows-machine-config-operator-bundle"},{"mirrors":["rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-rhel9-operator"],"source":"registry-proxy.engineering.redhat.com/openshift4-wincw/windows-machine-config-rhel9-operator"},{"mirrors":["rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-rhel9-operator"],"source":"registry.redhat.io/openshift4-wincw/windows-machine-config-rhel9-operator"},{"mirrors":["rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-operator-bundle"],"source":"registry.redhat.io/openshift4-wincw/windows-machine-config-operator-bundle"},{"mirrors":["rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000"],"source":"registry.redhat.io"}]}}
          creationTimestamp: "2025-03-26T08:54:37Z"
          generation: 1
          name: image-digest-mirror
          resourceVersion: "34214"
          uid: a55fc3c3-c090-4b2f-a470-66f375ba864f
        spec:
          imageDigestMirrors:
          - mirrors:
            - rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-operator-index
            source: registry-proxy.engineering.redhat.com/rh-osbs/iib
          - mirrors:
            - rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-operator-index
            - rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/rh-osbs/iib
            source: registry-proxy.engineering.redhat.com/rh-osbs/iib
          - mirrors:
            - rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-operator-bundle
            source: registry-proxy.engineering.redhat.com/openshift4-wincw/windows-machine-config-operator-bundle
          - mirrors:
            - rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-rhel9-operator
            source: registry-proxy.engineering.redhat.com/openshift4-wincw/windows-machine-config-rhel9-operator
          - mirrors:
            - rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-rhel9-operator
            source: registry.redhat.io/openshift4-wincw/windows-machine-config-rhel9-operator
          - mirrors:
            - rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-operator-bundle
            source: registry.redhat.io/openshift4-wincw/windows-machine-config-operator-bundle
          - mirrors:
            - rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000
            source: registry.redhat.io
      - apiVersion: config.openshift.io/v1
        kind: ImageDigestMirrorSet
        metadata:
          annotations:
            kubectl.kubernetes.io/last-applied-configuration: |
              {"apiVersion":"config.openshift.io/v1","kind":"ImageDigestMirrorSet","metadata":{"annotations":{},"name":"stage-repo"},"spec":{"imageDigestMirrors":[{"mirrors":["quay.io/redhat-user-workloads/windows-machine-conf-tenant/windows-machine-config-operator/windows-machine-config-operator-release-4.18"],"source":"registry.redhat.io/openshift4-wincw/windows-machine-config-rhel9-operator"},{"mirrors":["quay.io/redhat-user-workloads/windows-machine-conf-tenant/windows-machine-config-operator/windows-machine-config-operator-release-4.18"],"source":"registry.stage.redhat.io/openshift4-wincw/windows-machine-config-rhel9-operator"},{"mirrors":["quay.io/redhat-user-workloads/windows-machine-conf-tenant/windows-machine-config-operator/windows-machine-config-operator-bundle-release-4.18"],"source":"registry.redhat.io/openshift4-wincw/windows-machine-config-operator-bundle"},{"mirrors":["quay.io/redhat-user-workloads/windows-machine-conf-tenant/windows-machine-config-operator/windows-machine-config-operator-bundle-release-4.18"],"source":"registry.stage.redhat.io/openshift4-wincw/windows-machine-config-operator-bundle"}]}}
          creationTimestamp: "2025-03-26T09:27:32Z"
          generation: 1
          name: stage-repo
          resourceVersion: "51804"
          uid: cb3a8b52-e894-477e-992c-aeabd5ac449e
        spec:
          imageDigestMirrors:
          - mirrors:
            - quay.io/redhat-user-workloads/windows-machine-conf-tenant/windows-machine-config-operator/windows-machine-config-operator-release-4.18
            source: registry.redhat.io/openshift4-wincw/windows-machine-config-rhel9-operator
          - mirrors:
            - quay.io/redhat-user-workloads/windows-machine-conf-tenant/windows-machine-config-operator/windows-machine-config-operator-release-4.18
            source: registry.stage.redhat.io/openshift4-wincw/windows-machine-config-rhel9-operator
          - mirrors:
            - quay.io/redhat-user-workloads/windows-machine-conf-tenant/windows-machine-config-operator/windows-machine-config-operator-bundle-release-4.18
            source: registry.redhat.io/openshift4-wincw/windows-machine-config-operator-bundle
          - mirrors:
            - quay.io/redhat-user-workloads/windows-machine-conf-tenant/windows-machine-config-operator/windows-machine-config-operator-bundle-release-4.18
            source: registry.stage.redhat.io/openshift4-wincw/windows-machine-config-operator-bundle
      - apiVersion: config.openshift.io/v1
        kind: ImageDigestMirrorSet
        metadata:
          annotations:
            kubectl.kubernetes.io/last-applied-configuration: |
              {"apiVersion":"config.openshift.io/v1","kind":"ImageDigestMirrorSet","metadata":{"annotations":{},"name":"wmco-digested-mirror"},"spec":{"imageDigestMirrors":[{"mirrors":["rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-operator-index"],"source":"registry-proxy.engineering.redhat.com/rh-osbs/iib"}]}}
          creationTimestamp: "2025-03-26T09:27:33Z"
          generation: 1
          name: wmco-digested-mirror
          resourceVersion: "51844"
          uid: 65d8fa15-bae6-4d05-a9b3-e2e7974ad356
        spec:
          imageDigestMirrors:
          - mirrors:
            - rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-operator-index
            source: registry-proxy.engineering.redhat.com/rh-osbs/iib
      - apiVersion: config.openshift.io/v1
        kind: ImageDigestMirrorSet
        metadata:
          annotations:
            kubectl.kubernetes.io/last-applied-configuration: |
              {"apiVersion":"config.openshift.io/v1","kind":"ImageDigestMirrorSet","metadata":{"annotations":{},"name":"wmco-mirrors"},"spec":{"imageDigestMirrors":[{"mirrors":["rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-operator-bundle"],"source":"registry.redhat.io/openshift4-wincw/windows-machine-config-operator-bundle"},{"mirrors":["rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-rhel9-operator"],"source":"registry.redhat.io/openshift4-wincw/windows-machine-config-rhel9-operator"},{"mirrors":["rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-operator-index"],"source":"registry-proxy.engineering.redhat.com/rh-osbs/iib"},{"mirrors":["rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw"],"source":"brew.registry.redhat.io/rh-osbs"}]}}
          creationTimestamp: "2025-03-26T08:54:35Z"
          generation: 2
          name: wmco-mirrors
          resourceVersion: "48226"
          uid: 669068d4-bb1e-428e-8a06-af29f2ab788e
        spec:
          imageDigestMirrors:
          - mirrors:
            - rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-operator-bundle
            source: registry.redhat.io/openshift4-wincw/windows-machine-config-operator-bundle
          - mirrors:
            - rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-rhel9-operator
            source: registry.redhat.io/openshift4-wincw/windows-machine-config-rhel9-operator
          - mirrors:
            - rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw/windows-machine-config-operator-index
            source: registry-proxy.engineering.redhat.com/rh-osbs/iib
          - mirrors:
            - rrasouli-4096-bastion.mirror-registry.qe.devcluster.openshift.com:5000/openshift4-wincw
            source: brew.registry.redhat.io/rh-osbs
      kind: List
      metadata:
        resourceVersion: ""
      
      
      

              team-winc Team WinC
              rrasouli Aharon Rasouli
              None
              None
              Aharon Rasouli Aharon Rasouli
              None
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: