Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-36934

Hosted Control Planes Bare Metal - Proxy setting not applied to hosts.

XMLWordPrintable

    • None
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      Installed Hosted Control Planes on our environment, which are VMs, where we don't have access to VMWare vSphere, and therefore we use it like Baremetal over a own API. 
      The connectivity needs to go over an proxy to the outside.We installed an InfrasEnv "swisscom" and added the proxy settings to it.
      
      # oc get infraenvs.agent-install.openshift.io  -A
      NAMESPACE   NAME       ISO CREATED AT
      test        test       2024-06-28T13:54:50Z
      
      # oc get infraenvs.agent-install.openshift.io  test -n test -oyaml | grep -i proxy
        proxy:
          httpProxy: http://proxy-url:8080
          httpsProxy: http://proxy-url:8080
          noProxy: .cluster.local,.svc,10.0.0.0/16,10.128.0.0/14,127.0.0.1,172.30.0.0/16,localhost
      Then we added Hosts the dropdow menu "Add hosts" in the test "Host Inventory" by downloading the files in the "with iPXE" - by downloading the script - and adding it to the grub menu.
      
      # cat /etc/grub.d/50_iso#!/bin/bash
      cat <<EOFmenuentry 'Install HCP Nodes Local Files' --class fedora --class gnu-linux --class gnu --class os {kernel https://assisted-image-service-multicluster-engine.apps.example.com/boot-artifacts/kernel?arch=x86_64&version=4.15 
      
      initrd=initrd coreos.live.rootfs_url=https://assisted-image-service-multicluster-engine.apps.example.com/boot-artifacts/rootfs?arch=x86_64&version=4.15 random.trust_cpu=on rd.luks.options=discard ignition.firstboot ignition.platform.id=metal console=tty1 console=ttyS1,115200n8 coreos.inst.persistent-kargs="console=tty1 console=ttyS1,115200n8" ip=ipdetails rd.shell nameserver=ns1 nameserver=ns2 networkstatic=yes
      
      initrd --name initrd https://assisted-image-service-multicluster-engine.apps.example.com/images/621878e7-65b1-4b28-9be0-75f0b1934faf/pxe-initrd?api_key=eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbmZyYV9lbnZfaWQiOiI2MjE4NzhlNy02NWIxLTRiMjgtOWJlMC03NWYwYjE5MzRmYWYifQ.OSfMkMBcngI8HsNgfWfCnntWi2qvKjpsh_a0v0BD8hQ_NHw5Kowl9AONjCI-wT-hH4MjlGJSpOoMUrJL1fm1IA&arch=x86_64&version=4.15 ip=ipdetails rd.shell nameserver=ns2 nameserver=ns2 networkstatic=yesboot}EOF
      
      After booting, we had these hosts available in the test InfrasEnv and created a hosted control plane cluster "hcp-tst" - and added also the proxy settings to the cluster.
      
      But when adding this node to the cluster, we recognized, that the proxy setting are missing.
      
      under /etc/systemd/service the following files did not contain the proxy setting via EnvironmentFile=/etc/mco/proxy.envkubelet.service.d/10-mco-default-env.conf
      crio.service.d/10-mco-default-env.conf
      pivot.service.d/10-mco-default-env.conf
      rpm-ostreed.service.d/10-mco-default-env.conf
      nodeip-configuration.service
      machine-config-daemon-firstboot.service
      machine-config-daemon-pull.service
      
      We could get the host installed, after changing all proxy setting manually.
      Can you please tell us, why the proxy settings are not applied to the hosts  in the initrd and kernel files from the iPXE script ?
      
      Also we recognized, that kernel is not working in grub, but we have to use linux instead. 
      
      Also it was not possible to resolve the images from the url, since in grub the dns settings are missing and also the static ips. 
      
      We created an nmstate object, but this was not applied also for the setup:
      
      # oc get nmstateconfigs.agent-install.openshift.io -n test example.com -oyaml
      apiVersion: agent-install.openshift.io/v1beta1
      kind: NMStateConfig
      metadata:
        annotations:
          kubectl.kubernetes.io/last-applied-configuration: |
            removed
        creationTimestamp: "timestamp"
        generation: 1
        labels:
          infraenvs.agent-install.openshift.io: testinfra
        name: test
        namespace: test
      spec:
        config:
          dns-resolver:
            config:
              server:
              - dns-1
              - dns-2
          interfaces:
          - ipv4:
              address:
              - ip: ipv4
                prefix-length: 23
              enabled: true
            mac-address: 00:50:56:be:11:xx
            name: ens192
            state: up
            type: ethernet
          routes:
            config:
            - destination: 0.0.0.0/0
              next-hop-address: ipv4
              next-hop-interface: ens192
        interfaces:
        - macAddress: 00:50:56:be:11:xx
          name: ens192
      
      As an summary: our static ip node with proxy settings, was not able to be installed for our hcp cluster, because
      1. the iPXE script had kernel, whoich was not working (unknown command - we needed to use linux as command) and also DNS and static ip settings are not in the iPXE script. So we had to add this manually.
      2. the proxy settings from the InfrasEnv and cluster was not propagated to the host.
          

      Version-Release number of selected component (if applicable):

      4.14    

      How reproducible:

      N/A    

      Steps to Reproduce:

          1. steps mentioned in description
          2. added proxy to infraenv
          3. deployed HCP cluster
          

      Actual results:

       proxy was not propagated to the host   

      Expected results:

       proxy should have been added to host

      Additional info:

          

              agarcial@redhat.com Alberto Garcia Lamela
              rhn-support-chdeshpa Chinmay Deshpande
              Liangquan Li Liangquan Li
              Laura Hinson Laura Hinson
              Alberto Garcia Lamela
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated: