Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-74651

RHCOS 10 failures on metal ipv4 with attempted ipv6 access to registry.redhat.io

    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • Important
    • None
    • None
    • Proposed
    • None
    • In Progress
    • Release Note Not Required
    • None
    • None
    • None
    • None
    • None

      May need some help routing to the right component, this is a best first guess.

      When RHCOS 10 jobs went live we immediately found a large batch of systemic failures crossing OLM, Builds, and DeploymentConfigs. OLM and Deployment configs at least are traced back to what appears to be failing attempts to reach registry.redhat.io over IPv6, despite this being an ipv4 job. Unclear if Builds also do but it seems logical.

      We are unclear if the cluster is not properly being configured as ipv4, or if it's a problem with networking in general on RHEL 10.

      Sample job failure, the pattern appears about identical in every run, this same set of tests fail:

      https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-openshift-release-master-nightly-4.22-e2e-metal-ipi-ovn-ipv4-rhcos10-techpreview/2016770609702244352

      The deployment config tests show the problem most clearly in the test failure:

      [sig-apps][Feature:DeploymentConfig] deploymentconfigs with multiple image change triggers should run a successful deployment with multiple triggers [apigroup:apps.openshift.io][apigroup:image.openshift.io] [Suite:openshift/conformance/parallel] expand_less 	7s
      {  fail [github.com/openshift/origin/test/extended/deployments/deployments.go:552]: Unexpected error:
          <*fmt.wrapError | 0xc00260f4a0>: 
          Error running oc --namespace=e2e-test-cli-deployment-pcqxf --kubeconfig=/tmp/configfile1333795555 import-image registry.redhat.io/ubi8/ruby-30:latest --confirm --reference-policy=local:
          StdOut>
          error: tag latest failed: Internal error occurred: registry.redhat.io/ubi8/ruby-30:latest: Get "https://registry.redhat.io/v2/": dial tcp [2600:1f18:483:cf00:20a5:e8b3:1e40:90ab]:443: connect: network is unreachable
          imagestream.image.openshift.io/ruby-30 imported with errors
          
          Name:			ruby-30
          Namespace:		e2e-test-cli-deployment-pcqxf
          Created:		Less than a second ago
          Labels:			<none>
          Annotations:		openshift.io/image.dockerRepositoryCheck=2026-01-29T10:04:00Z
          Image Repository:	image-registry.openshift-image-registry.svc:5000/e2e-test-cli-deployment-pcqxf/ruby-30
          Image Lookup:		local=false
          Unique Images:		0
          Tags:			1
          
          latest
            tagged from registry.redhat.io/ubi8/ruby-30:latest
              prefer registry pullthrough when referencing this tag
          
            ! error: Import failed (InternalError): Internal error occurred: registry.redhat.io/ubi8/ruby-30:latest: Get "https://registry.redhat.io/v2/": dial tcp [2600:1f18:483:cf00:20a5:e8b3:1e40:90ab]:443: connect: network is unreachable
                Less than a second ago
      

      More job runs

              rhn-engineering-dgoodwin Devan Goodwin
              rhn-engineering-dgoodwin Devan Goodwin
              Jad Haj Yahya Jad Haj Yahya
              None
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: