When a cluster has a featuregate object created after a kubeletconfig object, the MCO creates the rendered ignition file from the featuregate object as n-1 of the rendered ignition for the kubeletconfig object. this causes the techpreview feature gates to be off     

    I am pretty sure 100%, we first saw this on the build02 cluster and I am working on verifying how reproducible 

    1. create a kubeletconfig mc
apiVersion: machineconfiguration.openshift.io/v1
kind: KubeletConfig
metadata:
  annotations:
    machineconfiguration.openshift.io/mc-name-suffix: ""
  name: set-pids-limit
spec:
  kubeletConfig:
    podPidsLimit: 8096
  machineConfigPoolSelector:
    matchExpressions:
    - key: pools.operator.machineconfiguration.openshift.io/worker
      operator: Exists
    2. create a featuregate object
apiVersion: config.openshift.io/v1
kind: FeatureGate
metadata:
  name: cluster
spec:
  featureSet: TechPreviewNoUpgrade     
  3. note the tech preview featuregates aren't set (UserNamespacesSupport, for example)
    

    tech preview featuregates are not set

    tech preview featuregates are set