-
Bug
-
Resolution: Unresolved
-
Undefined
-
None
-
1.8.0
-
None
-
RHDH Documentation 3283
-
Important
Description of problem:
Following the docs at https://docs.redhat.com/en/documentation/red_hat_developer_hub/1.8/html/accelerate_ai_development_with_openshift_ai_connector_for_red_hat_developer_hub/index reveals several issues:
1. Port 8080 Conflict (Critical)
The documentation fails to account for a port conflict between the rhoai-normalizer sidecar and the lightspeed-core sidecar.
- Issue: Both sidecars attempt to bind to port 8080. The rhoai-normalizer binds to :8080 for its metrics server, which prevents the lightspeed-core sidecar from binding ("address already in use").
- Missing Fix: The documentation must be updated to include the workaround, or disable metrics. The rhoai-normalizer sidecar configuration in the YAML needs the following arguments to change its metrics port:
- args: - '--metrics-address=:8081'
2. General Formatting and Typos
The documentation contains general errors that I worked around.
- Issue: The documentation contains YAML indentation and namespace errors.
One of the steps in the procedure is to create a:
"RoleBinding in the RHOAI namespace to grant the RHDH ServiceAccount read permissions to the model registry data (binding to registry-user-modelregistry-public)."
But the supplied RoleBinding has a namespace of ai-rhdh (which I am instructed to replace with my RHDH namespace). This RoleBinding should be put into the rhoai-model-registries namespace, no?
Next, the sidecar container definition has multiple YAML indentation issues (when copy/pasted). Here's it is copy-pasted from the docs, along with my commentary:
spec: template: spec: containers: - name: backstage-backend # This should not be here, as the main RHDH container is already defined - env: - name: NORMALIZER_FORMAT value: JsonArrayFormat - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace envFrom: - secretRef: name: rhdh-rhoai-connector-token image: quay.io/redhat-ai-dev/model-catalog-location-service@sha256:4f6ab6624a29f627f9f861cfcd5d18177d46aa2c67a81a75a1502c49bc2ff012 imagePullPolicy: Always name: location ports: - containerPort: 9090 name: location protocol: TCP volumeMounts: - mountPath: /opt/app-root/src/dynamic-plugins-root name: dynamic-plugins-root workingDir: /opt/app-root/src # workingDir should along with volumeMounts above - env: # This needs to be moved back to align with the previous "- env:" - name: NORMALIZER_FORMAT value: JsonArrayFormat - name: STORAGE_TYPE value: ConfigMap - name: BRIDGE_URL value: http://localhost:9090 - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace envFrom: - secretRef: name: rhdh-rhoai-connector-token image: quay.io/redhat-ai-dev/model-catalog-storage-rest@sha256:398095e7469e86d84b1196371286363f4b7668aa3e26370b4d78cb8d4ace1dc9 imagePullPolicy: Always name: storage-rest volumeMounts: - mountPath: /opt/app-root/src/dynamic-plugins-root name: dynamic-plugins-root workingDir: /opt/app-root/src # Same as above workingDir - env: - name: NORMALIZER_FORMAT value: JsonArrayFormat - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace envFrom: - secretRef: name: rhdh-rhoai-connector-token image: quay.io/redhat-ai-dev/model-catalog-rhoai-normalizer@sha256:fe6c05d57495d6217c4d584940ec552c3727847ff60f39f5d04f94be024576d8 imagePullPolicy: Always name: rhoai-normalizer volumeMounts: - mountPath: /opt/app-root/src/dynamic-plugins-root name: dynamic-plugins-root workingDir: /opt/app-root/src # same as above workingDir
- Issue: The documentation contains at least one typo.
In the example for ClusterRole rhdh-rhoai-connector it has an apiGroup of apiextensions.k8s.sio - this should be apiextensions.k8s.io (there is an extra s that should not be there and brings the whole thing down)
3. Misleading Log Output (Clarity Issue)
The documentation should proactively warn the user about alarming, but benign, error messages in the sidecar logs.
- Issue: The sidecar logs (e.g., in the location container) show errors that appear critical, such as:
-
- E1113 16:13:29.237865 1 k8s-client.go:53] in cluster config error: open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory
-
- E1113 16:13:29.258658 1 server.go:102] Get "http://10.128.2.58:7070/list": dial tcp 10.128.2.58:7070: connect: connection refused
- Missing Fix: While Gabe Montero noted these are "red herrings" and a disclaimer exists, the docs should be updated to be more specific. The disclaimer should explicitly state that errors like "connection refused" or "serviceaccount/token" may appear during startup and are not indicative of a failure, as long as the container eventually becomes healthy.
Prerequisites (if any, like setup, operators/versions):
Steps to Reproduce
Follow the docs.
Actual results:
YAML errors, permission issues (due to wrong apiGroups).
Expected results:
no errors, successful connection to RHOAI
Reproducibility (Always/Intermittent/Only Once):
Always
Build Details:
OCP 4.18
RHOAI 2.25.0
RHDH 1.8.0
