-
Bug
-
Resolution: Obsolete
-
Normal
-
None
-
4.14
-
Quality / Stability / Reliability
-
False
-
-
None
-
Important
-
No
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
When TechPreviewNoUpgrade is enabled on the management cluster, I can not create an hosted cluster successfully (aws platform in this failure)
Version-Release number of selected component (if applicable):
4.14
How reproducible:
100%
Steps to Reproduce:
1. create a management cluster with featureGate TechPreviewNoUpgrade 2. install hypershift operator and create a hosted cluster on the management cluster HO version: {"level":"info","ts":"2023-08-16T05:38:31Z","logger":"setup","msg":"Starting hypershift-operator-manager","version":"openshift/hypershift: 57b3cb46c41000033aaaa08480ed2256701c033b. Latest supported OCP: 4.14.0"} # install HO bin/hypershift install \ --hypershift-image=registry.build05.ci.openshift.org/ci-op-d1rbyhk6/pipeline@sha256:2ee89ab56aa2a48293ae462120a3b9afc7c2deb6bdaa9d3d5452d162a8f46d39 \ --oidc-storage-provider-s3-credentials=/var/run/secrets/ci.openshift.io/cluster-profile/.awscred \ --oidc-storage-provider-s3-bucket-name=0491705694d18d9fdaef \ --oidc-storage-provider-s3-region=us-east-1 \ --wait-until-available # create. /usr/bin/hypershift create cluster aws \ --image-content-sources /tmp/secret/mgmt_iscp.yaml \ --name 0491705694d18d9fdaef \ --node-pool-replicas 3 \ --instance-type m5.xlarge \ --base-domain qe.devcluster.openshift.com \ --region us-east-1 \ --control-plane-availability-policy HighlyAvailable \ --infra-availability-policy HighlyAvailable \ --pull-secret=/etc/ci-pull-credentials/.dockerconfigjson \ --aws-creds=/var/run/secrets/ci.openshift.io/cluster-profile/.awscred \ --release-image registry.build05.ci.openshift.org/ci-op-d1rbyhk6/release@sha256:8a5507bf897252cab6d1957d9477bce45e7427f4f798450605d3503aed936594 \ --additional-tags=expirationDate=2023-08-16T09:39+00:00
Actual results:
Here are some error logs in HO: {"level":"error","ts":"2023-08-16T05:59:06Z","msg":"Failed to reconcile NodePool","controller":"nodepool","controllerGroup":"hypershift.openshift.io","controllerKind":"NodePool","NodePool":{"name":"0491705694d18d9fdaef-us-east-1a","namespace":"clusters"},"namespace":"clusters","name":"0491705694d18d9fdaef-us-east-1a","reconcileID":"2e9d0bbe-2ed4-4e76-97df-e0ae09396fc2","error":"admission webhook \"validation.awsmachinetemplate.infrastructure.cluster.x-k8s.io\" denied the request: AWSMachineTemplate.infrastructure.cluster.x-k8s.io \"0491705694d18d9fdaef-us-east-1a\" is invalid: spec.template.spec.cloudInit.secureSecretsBackend: Forbidden: cannot be set if spec.template.spec.cloudInit.insecureSkipSecretsManager is true","stacktrace":"github.com/openshift/hypershift/hypershift-operator/controllers/nodepool.(*NodePoolReconciler).Reconcile\n\t/hypershift/hypershift-operator/controllers/nodepool/nodepool_controller.go:215\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/hypershift/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:121\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/hypershift/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:320\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/hypershift/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:273\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/hypershift/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:234"}
Expected results:
The hosted cluster can be created successfully
Additional info:
slack discussion:
https://redhat-internal.slack.com/archives/C01C8502FMM/p1692167982285039
From the slack discussion:
"so we (Hypershift) don’t install the cluster-api-provider-aws webhook, which explains why we didn’t see this error before.
But something here is installing the webhook on the mgmt cluster https://github.com/openshift/cluster-api-provider-aws/blob/master/config/webhook/manifests.yaml"