Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-43557

In OCB. New MOSBs are marked as failed if there are previously failed MOSBs

XMLWordPrintable

    • Moderate
    • None
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      When we create a MOSC with secrets containing bad information a build is triggered and this build fails.
      
      If we update the secrets with the right information and we create a new MC to force a new build, the new build's pod succeeds but the MOSB resource is marked as failed. 
      
      The reason seems to be a wrong interaction with the previously existing failed MOSB.
      
      Moreover, the failed MOSB is not garbage collected if remove the MOSC reosource.
          

      Version-Release number of selected component (if applicable):

      4.18.0-0.nightly-2024-10-17-225959
      
      It happens in 4.17.0-0.nightly-2024-10-17-205717 too.
          

      How reproducible:

      Always
          

      Steps to Reproduce:

          1. Create a custom pool
          
          $ oc create -f - << EOF
      apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfigPool
      metadata:
        name: infra
      spec:
        machineConfigSelector:
          matchExpressions:
            - {key: machineconfiguration.openshift.io/role, operator: In, values: [worker,infra]}
        nodeSelector:
          matchLabels:
            node-role.kubernetes.io/infra: ""
      EOF
      
          2.Create 3 legacy secrets:
      $ oc create secret generic my-input-pull --from-literal '.dockercfg={"fake-repo":{"auth":"'$(echo -n fake-pull-pass | base64)'"}}' --type=kubernetes.io/dockercfg
      secret/my-input-pull created
      
      $ oc create secret generic my-input-push --from-literal '.dockercfg={"fake-repo":{"auth":"'$(echo -n fake-push-pass | base64)'"}}' --type=kubernetes.io/dockercfg
      secret/my-input-push created
      
      $ oc create secret generic my-output-pull --from-literal '.dockercfg={"fake-repo":{"auth":"'$(echo -n fake-output-pull-pass | base64)'"}}' --type=kubernetes.io/dockercfg
      secret/my-output-pull created
          
          
          3.Create a MSOC using those secrets
          
          $ oc create -f - << EOF
      apiVersion: machineconfiguration.openshift.io/v1alpha1
      kind: MachineOSConfig
      metadata:
        name: infra
      spec:
        machineConfigPool:
          name: infra
        buildOutputs:
          currentImagePullSecret:
            name: my-output-pull
        buildInputs:
          imageBuilder:
            imageBuilderType: PodImageBuilder
          baseImagePullSecret:
            name:  my-input-pull
          renderedImagePushSecret:
            name: my-input-push
          renderedImagePushspec: "image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/ocb-image:latest"
      EOF
      
      
          4. MachineOSBuild should fail
      
      5. Update all the secrets with the right values
      $ oc -n openshift-machine-config-operator set data secret/my-input-pull --from-literal ".dockercfg=$(oc get secret pull-secret -n openshift-config '--template={{index .data ".dockerconfigjson" | base64decode}}' | jq .auths)"
      secret/my-input-pull data updated
      
      $  oc -n openshift-machine-config-operator set data secret/my-input-push --from-literal ".dockercfg=$(oc get secret  -n openshift-machine-config-operator $(oc get -n openshift-machine-config-operator sa builder -ojsonpath='{.secrets[0].name}') '--template={{index .data ".dockercfg" | base64decode}}')"
      secret/my-input-pull data updated
      
      
      $ oc -n openshift-machine-config-operator set data secret/my-output-pull --from-literal ".dockercfg=$(oc get secret  -n openshift-machine-config-operator $(oc get -n openshift-machine-config-operator sa default -ojsonpath='{.secrets[0].name}') '--template={{index .data ".dockercfg" | base64decode}}')"
      secret/my-output-pull data updated
      
      
      6. Create a new machineconfig
      
      $ oc create -f - << EOF
      apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfig
      metadata:
        labels:
          machineconfiguration.openshift.io/role: infra
        name: tc-74111-mco-canonicalized-secrets
      spec:
        config:
          ignition:
            version: 3.1.0
          storage:
            files:
            - contents:
                source: data:text/plain;charset=utf-8;base64,dGVzdA==
              mode: 420
              path: /etc/test-tc-74111-mco-canonicalized-secrets.test
      EOF
      
      
      

      Actual results:

          The new MOSB should be successful because its secrets contain the right information. Nevertheless, the new build is marked as failed.
          
          

      Expected results:

          Once we fix the content in the secrets, the new MOSBs should succeed.
      
          

      Additional info:

      Moreover, if we remove the MOSC resource after the second build finishes, we can see that the first failed MOSB is not garbage collected and is leaked.
      
      Workaround: If we remove the failed pod that belongs to the first MOSB before executing the second MOSB everything works without problems.     

              team-mco Team MCO
              sregidor@redhat.com Sergio Regidor de la Rosa
              Sergio Regidor de la Rosa Sergio Regidor de la Rosa
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: