Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-58191

Image Mode constantly creating new MOSBs when using a non-dockerconfigjson push secret

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Important
    • No
    • None
    • Proposed
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      
      When in Image Mode we configure the MOSC resource with a non-dockerconfigjson renderedImagePushSecret the os-biulder pod will create a new MOSB everytime it is restarted. Since the os-builder is drained when an image is applied, it constantly creates new MOSB resources continuously creating new images and restarting the nodes.
      
      

      Version-Release number of selected component (if applicable):

      4.20.0-0.nightly-2025-06-27-040027
      
      But it should impact 4.19 and 4.18 too.
      

      How reproducible:

      Always
      

      Steps to Reproduce:

      
      Enable Image Mode like this:
      
      oc create -f - << EOF
      apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineOSConfig
      metadata:
        name: worker-mosc
      spec:
        machineConfigPool:
          name: worker
        imageBuilder:
          imageBuilderType: Job
        baseImagePullSecret:
          name: $(oc get secret -n openshift-config pull-secret -o json | jq "del(.metadata.namespace, .metadata.creationTimestamp, .metadata.resourceVersion, .metadata.uid, .metadata.name)" | jq '.metadata.name="pull-copy"' | oc -n openshift-machine-config-operator create -f - &> /dev/null; echo -n "pull-copy")
        renderedImagePushSecret:
          name: $(oc get -n openshift-machine-config-operator sa builder -ojsonpath='{.secrets[0].name}')
        renderedImagePushSpec: "image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/ocb-image:latest"
        containerFile:
            - content: |-
                RUN touch /etc/pre-upgrade.test
      EOF
      
      
      
      

      Actual results:

      The image is built, MCO applies the image, while applying the image the os-builder is evicted, when the new os-builder pod starts running fails with this error
      I0627 09:07:36.810557       1 reconciler.go:626] Existing MachineOSBuild "worker-mosc-03ac0cc443fd1846819f6fc9b452dc73" found, checking if image "image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/ocb-image
      @sha256:622baa7bbbea2c0337089280e3df8159997f0472f5ddd13fcd24231d7f0a9ffd" still exists
      W0627 09:07:36.814474       1 reconciler.go:636] Could not get auth file path: rendered push secret for MachineOSConfig "worker-mosc" is not of type "kubernetes.io/dockerconfigjson", cannot delete image
      W0627 09:07:36.814582       1 reconciler.go:636] Could not get auth file path: rendered push secret for MachineOSConfig "worker-mosc" is not of type "kubernetes.io/dockerconfigjson", cannot delete image
      I0627 09:07:36.857187       1 reconciler.go:646] Existing MachineOSBuild image "image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/ocb-image:worker-mosc-03ac0cc443fd1846819f6fc9b452dc73" no longer exists, skipping reuse. Got error: error getting image source "docker://image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/ocb-image:worker-mosc-03ac0cc443fd1846819f6fc9b452dc73": reading manifest worker-mosc-03ac0cc443fd1846819f6fc9b452dc73 in image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/ocb-image: authentication required
      I0627 09:07:36.857206       1 reconciler.go:650] Deleting MachineOSBuild "worker-mosc-03ac0cc443fd1846819f6fc9b452dc73" so we can rebuild it to create a new image
      I0627 09:07:36.857188       1 reconciler.go:646] Existing MachineOSBuild image "image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/ocb-image:worker-mosc-03ac0cc443fd1846819f6fc9b452dc73" no longer exists, skipping reuse. Got error: error getting image source "docker://image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/ocb-image:worker-mosc-03ac0cc443fd1846819f6fc9b452dc73": reading manifest worker-mosc-03ac0cc443fd1846819f6fc9b452dc73 in image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/ocb-image: authentication required
      I0627 09:07:36.857321       1 reconciler.go:650] Deleting MachineOSBuild "worker-mosc-03ac0cc443fd1846819f6fc9b452dc73" so we can rebuild it to create a new image
      I0627 09:07:37.904751       1 reconciler.go:488] New MachineOSBuild created: worker-mosc-03ac0cc443fd1846819f6fc9b452dc73
      
      
      
      And after this failure the os-builder pod created a new MOSB resource because it couldn't verify that the image actually existed.
      
      The builder pod can push the image without problems, though, because it is prepared to handle non-dockerconfigjson push secrets. 
      
      

      Expected results:

      
      MOSB resources shouldnt be created in a loop
      
      

      Additional info:

      
      
      

              zzlotnik@redhat.com Zack Zlotnik
              sregidor@redhat.com Sergio Regidor de la Rosa
              None
              None
              Sergio Regidor de la Rosa Sergio Regidor de la Rosa
              None
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated: