Overview:

During investigation of a different issue we noticed a problematic behavior of our operator, specifically the Helm reconciler side: During execution of our pre-Extensions, specifically the extension responsible for setting defaulting annotations, the custom resource is patched on the cluster, as part of which it is also re-retrieved. This fresh version of the CR will currently be converted back into an unstructured object using runtime.DefaultUnstructuredConverter.ToUnstructured(). This step has the undesired side-effect that some field normalization is applied, for example to

spec.central.db.resources.limits.cpu

When the cluster has stored:

spec:
  central:
    db:
      resources:
        limits:
          cpu: 1 # Note, this is a number.

then the in-memory object before execution of the extension contains the same. But after conversion using ToUnstructured(), this is silently normalized to

spec:
  central:
    db:
      resources:
        limits:
          cpu: "1" # Note, this is a string.

There are also a couple of other silent normalizations happening involving "1" and "1000m".

Now, previously this also happened but we neither did we know about this nor did we had a reason to care about this. The situation has changed now with the introduction of the "aggressive conflict resolution", as part of which we need to make sure that we don't accidentally overwrite any spec changes which have might have been written while the helm-operator reconciler was executing. For this we have a specific check for spec-mismatches. This check was failing and prevented successful conflict resolution.

Solution

We need to modify our defaulting extension to not modify the in-memory spec representation.