-
Bug
-
Resolution: Not a Bug
-
Major
-
None
-
4.14.z
-
Quality / Stability / Reliability
-
False
-
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
Installed Hosted Control Planes on our environment, which are VMs, where we don't have access to VMWare vSphere, and therefore we use it like Baremetal over a own API.
The connectivity needs to go over an proxy to the outside.We installed an InfrasEnv "swisscom" and added the proxy settings to it.
# oc get infraenvs.agent-install.openshift.io -A
NAMESPACE NAME ISO CREATED AT
test test 2024-06-28T13:54:50Z
# oc get infraenvs.agent-install.openshift.io test -n test -oyaml | grep -i proxy
proxy:
httpProxy: http://proxy-url:8080
httpsProxy: http://proxy-url:8080
noProxy: .cluster.local,.svc,10.0.0.0/16,10.128.0.0/14,127.0.0.1,172.30.0.0/16,localhost
Then we added Hosts the dropdow menu "Add hosts" in the test "Host Inventory" by downloading the files in the "with iPXE" - by downloading the script - and adding it to the grub menu.
# cat /etc/grub.d/50_iso#!/bin/bash
cat <<EOFmenuentry 'Install HCP Nodes Local Files' --class fedora --class gnu-linux --class gnu --class os {kernel https://assisted-image-service-multicluster-engine.apps.example.com/boot-artifacts/kernel?arch=x86_64&version=4.15
initrd=initrd coreos.live.rootfs_url=https://assisted-image-service-multicluster-engine.apps.example.com/boot-artifacts/rootfs?arch=x86_64&version=4.15 random.trust_cpu=on rd.luks.options=discard ignition.firstboot ignition.platform.id=metal console=tty1 console=ttyS1,115200n8 coreos.inst.persistent-kargs="console=tty1 console=ttyS1,115200n8" ip=ipdetails rd.shell nameserver=ns1 nameserver=ns2 networkstatic=yes
initrd --name initrd https://assisted-image-service-multicluster-engine.apps.example.com/images/621878e7-65b1-4b28-9be0-75f0b1934faf/pxe-initrd?api_key=eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbmZyYV9lbnZfaWQiOiI2MjE4NzhlNy02NWIxLTRiMjgtOWJlMC03NWYwYjE5MzRmYWYifQ.OSfMkMBcngI8HsNgfWfCnntWi2qvKjpsh_a0v0BD8hQ_NHw5Kowl9AONjCI-wT-hH4MjlGJSpOoMUrJL1fm1IA&arch=x86_64&version=4.15 ip=ipdetails rd.shell nameserver=ns2 nameserver=ns2 networkstatic=yesboot}EOF
After booting, we had these hosts available in the test InfrasEnv and created a hosted control plane cluster "hcp-tst" - and added also the proxy settings to the cluster.
But when adding this node to the cluster, we recognized, that the proxy setting are missing.
under /etc/systemd/service the following files did not contain the proxy setting via EnvironmentFile=/etc/mco/proxy.envkubelet.service.d/10-mco-default-env.conf
crio.service.d/10-mco-default-env.conf
pivot.service.d/10-mco-default-env.conf
rpm-ostreed.service.d/10-mco-default-env.conf
nodeip-configuration.service
machine-config-daemon-firstboot.service
machine-config-daemon-pull.service
We could get the host installed, after changing all proxy setting manually.
Can you please tell us, why the proxy settings are not applied to the hosts in the initrd and kernel files from the iPXE script ?
Also we recognized, that kernel is not working in grub, but we have to use linux instead.
Also it was not possible to resolve the images from the url, since in grub the dns settings are missing and also the static ips.
We created an nmstate object, but this was not applied also for the setup:
# oc get nmstateconfigs.agent-install.openshift.io -n test example.com -oyaml
apiVersion: agent-install.openshift.io/v1beta1
kind: NMStateConfig
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
removed
creationTimestamp: "timestamp"
generation: 1
labels:
infraenvs.agent-install.openshift.io: testinfra
name: test
namespace: test
spec:
config:
dns-resolver:
config:
server:
- dns-1
- dns-2
interfaces:
- ipv4:
address:
- ip: ipv4
prefix-length: 23
enabled: true
mac-address: 00:50:56:be:11:xx
name: ens192
state: up
type: ethernet
routes:
config:
- destination: 0.0.0.0/0
next-hop-address: ipv4
next-hop-interface: ens192
interfaces:
- macAddress: 00:50:56:be:11:xx
name: ens192
As an summary: our static ip node with proxy settings, was not able to be installed for our hcp cluster, because
1. the iPXE script had kernel, whoich was not working (unknown command - we needed to use linux as command) and also DNS and static ip settings are not in the iPXE script. So we had to add this manually.
2. the proxy settings from the InfrasEnv and cluster was not propagated to the host.
Version-Release number of selected component (if applicable):
4.14
How reproducible:
N/A
Steps to Reproduce:
1. steps mentioned in description
2. added proxy to infraenv
3. deployed HCP cluster
Actual results:
proxy was not propagated to the host
Expected results:
proxy should have been added to host
Additional info: