Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-4160

Fails to deprovision cluster when swift omits 'content-type'

    XMLWordPrintable

Details

    • +
    • Low
    • ShiftStack Sprint 228
    • 1
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Hide
      Some object storage instances respond with "204 No Content" when there is no content to display. The OpenStack SDK we use in OpenShift does not handle 204s correctly. With this change, the Installer works around the issue when there are zero items to list.
      Show
      Some object storage instances respond with "204 No Content" when there is no content to display. The OpenStack SDK we use in OpenShift does not handle 204s correctly. With this change, the Installer works around the issue when there are zero items to list.
    • Bug Fix

    Description

      Description of problem: This is a follow-up to OCPBUGS-2795 and OCPBUGS-2941.

      The installer fails to destroy the cluster when the OpenStack object storage omits 'content-type' from responses. This can happen on responses with HTTP status code 204, where a reverse proxy is truncating content-related headers (see this nginX bug report). In such cases, the Installer errors with:

      level=error msg=Bulk deleting of container "5ifivltb-ac890-chr5h-image-registry-fnxlmmhiesrfvpuxlxqnkoxdbl" objects failed: Cannot extract names from response with content-type: []
      

      Listing container object suffers from the same issue as listing the containers and this one isn't fixed in latest versions of gophercloud. I've reported https://github.com/gophercloud/gophercloud/issues/2509 and fixing it with https://github.com/gophercloud/gophercloud/issues/2510, however we likely won't be able to backport the bump to gophercloud master back to release-4.8 so we'll have to look for alternatives.

      I'm setting the priority to critical as it's causing all our jobs to fail in master.

      Version-Release number of selected component (if applicable):

      4.8.z
      

      How reproducible:

      Likely not happening in customer environments where Swift is exposed directly. We're seeing the issue in our CI where we're using a non-RHOSP managed cloud.
      

      Steps to Reproduce:

      1.
      2.
      3.
      

      Actual results:

       

      Expected results:

       

      Additional info:

       

      Attachments

        Issue Links

          Activity

            People

              pprinett@redhat.com Pierre Prinetti
              openshift-crt-jira-prow OpenShift Prow Bot
              Jon Uriarte Jon Uriarte
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: