Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-5154

OCP on OSP - Image registry is deployed with cinder instead of swift storage backend

    XMLWordPrintable

Details

    • ?
    • Important
    • ShiftStack Sprint 230
    • Rejected
    • Hide

      None

      Show
      None
    • Hide
      * Previously, the `cluster-image-registry-operator` would default to using persistent volume claim (PVC) when it failed to reach Swift. With this update, failure to connect to {rh-openstack-first} API or other incidental failures cause the `cluster-image-registry-operator` to retry the probe. During the retry, the default to PVC only occurs if the {rh-openstack} catalog is correctly found, and it does not contain object storage; or alternatively, if {rh-openstack} catalog is there and the current user does not have permission to list containers. (link:https://issues.redhat.com/browse/OCPBUGS-5154[*OCPBUGS-5154*])
      Show
      * Previously, the `cluster-image-registry-operator` would default to using persistent volume claim (PVC) when it failed to reach Swift. With this update, failure to connect to {rh-openstack-first} API or other incidental failures cause the `cluster-image-registry-operator` to retry the probe. During the retry, the default to PVC only occurs if the {rh-openstack} catalog is correctly found, and it does not contain object storage; or alternatively, if {rh-openstack} catalog is there and the current user does not have permission to list containers. (link: https://issues.redhat.com/browse/OCPBUGS-5154 [* OCPBUGS-5154 *])

    Description

      This is a clone of issue OCPBUGS-4090. The following is the description of the original issue:
      —
      Description of problem:

      While deploying the OCP cluster on OSP 16, the installer was supposed to use swift as a storage backend for image-registry however it uses Cinder instead.

This can be seen when installing an OCP 4.8 cluster on OSP 16. The swiftoperator role is already assigned to the OpenStack user.

Below error was found in the registry operator logs related to "swift":
~~~
$ oc logs cluster-image-registry-operator-7bbdcfb94c-lrqjs | grep swift
E1116 09:43:06.539763       1 swift.go:67] swift storage inaccessible: Failed to authenticate provider client: Post "https://osp.ipz001.internal.bosch.cloud:13000/v3/auth/tokens": dial tcp 10.140.249.17:13000: connect: connection timed out
~~~

Also when curled from the registry operator pod, it gets connected, So it seems it was just not ready at the time of installation. Now connectivity seems to be fine.

~~~
sh-4.4$ curl -vk https://osp.ipz001.internal.bosch.cloud:13000/v3/auth/tokens

* Uses proxy env variable NO_PROXY == '.bosch.com,.cluster.local,.svc,.webapp.inside.bosch.cloud,10.140.180.0/23,10.140.214.0/24,10.140.249.0/24,10.140.250.30,10.140.253.2,10.140.254.0/24,10.40.0.0/24,127.0.0.1,169.254.169.254,192.168.0.0/17,192.168.128.0/17,api-int.de1qua.osh.ipz001.internal.bosch.cloud,bcr-de01.inside.bosch.cloud,internal.bosch.cloud,localhost,osh.ipz001.internal.bosch.cloud'
*   Trying 10.140.249.17...
* TCP_NODELAY set
* Connected to osp.ipz001.internal.bosch.cloud (10.140.249.17) port 13000 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
~~~

As a Day-2 operation, switching the storage backend to swift works but with this action, all the images that are stored already in the registry gets deleted.

This might be related to https://issues.redhat.com/browse/OCPBUGS-2941 or https://issues.redhat.com/browse/OCPBUGS-2795

      Version-Release number of selected component (if applicable):

      4.10.28

      Additional info:

      We have started discussion with the engineering team on a slack thread and as well as over the email. https://coreos.slack.com/archives/CH98TDJUD/p1668097534123929

      Attachments

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. OCPBUGS-5778 OCP on OSP - Image registry is deployed with cinder instead of swift storage backend

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          clones

          Bug - A problem which impairs or prevents the functions of the product. OCPBUGS-4090 OCP on OSP - Image registry is deployed with cinder instead of swift storage backend

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. OCPBUGS-4090 OCP on OSP - Image registry is deployed with cinder instead of swift storage backend

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is cloned by

          Bug - A problem which impairs or prevents the functions of the product. OCPBUGS-5778 OCP on OSP - Image registry is deployed with cinder instead of swift storage backend

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          links to

          GitHub openshift/cluster-image-registry-operator#826: [release-4.12] OCPBUGS-5154: swift: Retry connecting to OpenStack

          Activity

            People

              pprinett@redhat.com Pierre Prinetti
              openshift-crt-jira-prow OpenShift Prow Bot
              Jon Uriarte Jon Uriarte
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: