Description of problem:

I try to create new bare metal SNO cluster from ACM with host discovery. It fails to give me image download url, so i suppose it fails to build the image.

 

Version-Release number of selected component (if applicable):

advanced-cluster-management.v2.7.0

How reproducible:

Has happened earlier too. I've never managed to finish the cluster install for different reasons.

Steps to Reproduce:

1. Create new cluster, select discovered hosts
2. accept all the things, give domain and cluster name... next next
3. .

Actual results:

Expected results:

To have downloadable image. At earlier try, I got help from the AI slack. I don't recall exactly the steps, but I remember the ACM didn't have image url list present, and we added it somewhere. List is here:

https://github.com/openshift/assisted-service/blob/0891c15395f5ae7e53b983e5d2971ed6dc8909b1/data/default_os_images.json#L51-L54

Also there was some need to change image type to or from multi

 

Additional info:

Install target is Intel NUC this time, it shouldn't matter.

 

InfraEnv:

▶ oc get -o yaml infraenv fi-sno-01
apiVersion: agent-install.openshift.io/v1beta1
kind: InfraEnv
metadata:
  creationTimestamp: "2023-03-02T08:59:44Z"
  finalizers:
  - infraenv.agent-install.openshift.io/ai-deprovision
  generation: 3
  labels:
    networkType: dhcp
  name: fi-sno-01
  namespace: fi-sno-01
  resourceVersion: "1193491198"
  uid: 424a20de-9321-4a79-849c-78f7720f0867
spec:
  agentLabels:
    agentBareMetal-generated-infraenv-ai-flow: fi-sno-01-fi-sno-01
  clusterRef:
    name: fi-sno-01
    namespace: fi-sno-01
  cpuArchitecture: x86_64
  ipxeScriptType: DiscoveryImageAlways
  nmStateConfigLabelSelector: {}
  pullSecretRef:
    name: pullsecret-cluster-fi-sno-01
  sshAuthorizedKey: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHV4O/qEctm+YmAY/o0aKk+sgxbFhLMsNN1m4j8FNKSF
    admin@coollab
status:
  agentLabelSelector:
    matchLabels:
      infraenvs.agent-install.openshift.io: fi-sno-01
  bootArtifacts:
    initrd: https://assisted-image-service-multicluster-engine.apps.ocpmgt.cool.lab/images/688ee819-9ede-4284-ace7-2954b179247c/pxe-initrd?api_key=eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbmZyYV9lbnZfaWQiOiI2ODhlZTgxOS05ZWRlLTQyODQtYWNlNy0yOTU0YjE3OTI0N2MifQ.g4yqM9AJLLk_x23w6g5va_KRKjjqNpnoNRFqawxRszZ7-XiiWV_24PEhFMHYCzLEpEWdUIhNU96EthgC_X53_Q&arch=x86_64&version=4.12.2
    ipxeScript: https://assisted-service-multicluster-engine.apps.ocpmgt.cool.lab/api/assisted-install/v2/infra-envs/688ee819-9ede-4284-ace7-2954b179247c/downloads/files?api_key=eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbmZyYV9lbnZfaWQiOiI2ODhlZTgxOS05ZWRlLTQyODQtYWNlNy0yOTU0YjE3OTI0N2MifQ.t1KkKnpCpPa25CW_ET3x2gcnycmsPWqr8FMRavIECZFO0Trr-GnY1hPo0LSU8BGhihPUuaSJMRxFvy2wcLz8rQ&file_name=ipxe-script
    kernel: https://assisted-image-service-multicluster-engine.apps.ocpmgt.cool.lab/boot-artifacts/kernel?arch=x86_64&version=4.12.2
    rootfs: https://assisted-image-service-multicluster-engine.apps.ocpmgt.cool.lab/boot-artifacts/rootfs?arch=x86_64&version=4.12.2
  conditions:
  - lastTransitionTime: "2023-03-02T11:36:54Z"
    message: 'Failed to create image due to an internal error: The requested RHCOS
      version (4.12, arch: x86_64) does not have a matching OpenShift release image'
    reason: ImageCreationError
    status: "False"
    type: ImageCreated
  debugInfo:
    eventsURL: https://assisted-service-multicluster-engine.apps.ocpmgt.cool.lab/api/assisted-install/v2/events?api_key=eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbmZyYV9lbnZfaWQiOiI2ODhlZTgxOS05ZWRlLTQyODQtYWNlNy0yOTU0YjE3OTI0N2MifQ.8j7dt-RiRznf_C6vmKIkSY25cLmAuJopJVhKfLOCZjYnbcx-_QTiq2hwwIMKnKRLOmpPJpf_WeWwWikS8-F97g&infra_env_id=688ee819-9ede-4284-ace7-2954b179247c

 

▶ oc logs -n multicluster-engine assisted-image-service-0                          
{{

{"file":"/remote-source/app/pkg/imagestore/imagestore.go:298","func":"github.com/openshift/assisted-image-service/pkg/imagestore.(*rhcosStore).cleanDataDir","level":"info","msg":"Removing /data/rhcos-minimal-iso-4.12.2-412.86.202301311551-0-x86_64.iso from data directory","time":"2023-02-22T23:21:35Z"}

}}
{{

{"file":"/remote-source/app/pkg/servers/servers.go:87","func":"github.com/openshift/assisted-image-service/pkg/servers.(*ServerInfo).httpListen","level":"info","msg":"Starting http handler on :8081...","time":"2023-02-22T23:21:35Z"}

}}
{{

{"file":"/remote-source/app/pkg/servers/servers.go:94","func":"github.com/openshift/assisted-image-service/pkg/servers.(*ServerInfo).httpsListen","level":"info","msg":"Starting https handler on :8080...","time":"2023-02-22T23:21:35Z"}

}}
{{

{"file":"/remote-source/app/pkg/imagestore/imagestore.go:231","func":"github.com/openshift/assisted-image-service/pkg/imagestore.(*rhcosStore).Populate","level":"info","msg":"Creating minimal iso for 4.12.2-412.86.202301311551-0-x86_64","time":"2023-02-22T23:21:35Z"}

}}
{{

{"file":"/remote-source/app/pkg/imagestore/imagestore.go:244","func":"github.com/openshift/assisted-image-service/pkg/imagestore.(*rhcosStore).Populate","level":"info","msg":"Finished creating minimal iso for 4.12.2-x86_64 (412.86.202301311551-0)","time":"2023-02-22T23:21:43Z"}

}}
{{

{"file":"/remote-source/app/internal/handlers/readiness.go:40","func":"github.com/openshift/assisted-image-service/internal/handlers.(*ReadinessHandler).Enable","level":"info","msg":"API is enabled","time":"2023-02-22T23:21:43Z"}

}}
2023/02/25 06:21:26 http: TLS handshake error from 10.134.2.1:52936: read tcp 10.133.2.17:8080->10.134.2.1:52936: read: connection reset by peer

 

▶ oc logs -n multicluster-engine assisted-service-7fd67c6df8-7pfr5|grep -i 'error|warning'

time="2023-03-02T13:43:18Z" level=debug msg="cleanup up ISODownloadURL due to  due to an internal error: The requested RHCOS version (4.12, arch: x86_64) does not have a matching OpenShift release image" func="github.com/openshift/assisted-service/internal/controller/controllers.(*InfraEnvReconciler).handleEnsureISOErrors" file="/remote-source/assisted-service/app/internal/controller/controllers/infraenv_controller.go:718" go-id=1404 infra_env=fi-sno-01 infra_env_namespace=fi-sno-01 request_id=b0716bc8-eca4-4bd5-b563-731c377e60bc
time="2023-03-02T13:43:19Z" level=error msg="failed to update InfraEnv" func="github.com/openshift/assisted-service/internal/controller/controllers.(*InfraEnvReconciler).ensureISO" file="/remote-source/assisted-service/app/internal/controller/controllers/infraenv_controller.go:390" error="The requested RHCOS version (4.12, arch: x86_64) does not have a matching OpenShift release image" go-id=1404 infra_env=fi-sno-02 infra_env_namespace=fi-sno-02 request_id=b35d1cbb-4ef9-44da-b35b-fe93d79582d9
time="2023-03-02T13:43:19Z" level=error msg="infraEnv reconcile failed" func="github.com/openshift/assisted-service/internal/controller/controllers.(*InfraEnvReconciler).handleEnsureISOErrors" file="/remote-source/assisted-service/app/internal/controller/controllers/infraenv_controller.go:701" error="The requested RHCOS version (4.12, arch: x86_64) does not have a matching OpenShift release image" go-id=1404 infra_env=fi-sno-02 infra_env_namespace=fi-sno-02 request_id=b35d1cbb-4ef9-44da-b35b-fe93d79582d9
time="2023-03-02T13:43:19Z" level=debug msg="cleanup up ISODownloadURL due to  due to an internal error: The requested RHCOS version (4.12, arch: x86_64) does not have a matching OpenShift release image" func="github.com/openshift/assisted-service/internal/controller/controllers.(*InfraEnvReconciler).handleEnsureISOErrors" file="/remote-source/assisted-service/app/internal/controller/controllers/infraenv_controller.go:718" go-id=1404 infra_env=fi-sno-02 infra_env_namespace=fi-sno-02 request_id=b35d1cbb-4ef9-44da-b35b-fe93d79582d9