Loading...

Type: Bug
Resolution: Done
Priority: Blocker
Fix Version/s: 3.5.0.GA
Affects Version/s: 3.5.0.GA
Component/s: productization: build & internals, Team A: Dashboard, devfile and plugin registries, open-vsx, devfile-converter, configbump + traefik, image-puller, server, gateway, authentication, try-in-web-ide action, telemetry
Labels:
- regression

Blocked:
False
Blocked Reason:

Hide

None

Show
None
Ready:
False
Intelligence Requested:
Market:

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

Test DS 3.5.0.RC-03-15 instance installed from staging: https://devspaces.apps.ocp410.crw-qe.com/
Test OCP 4.10 instance: https://console-openshift-console.apps.ocp410.crw-qe.com/

Description of problem:

There is inconsistency in DS 3.5.0.RC-03-15 metadata: different samples has different udi-rhel8 image digests:

it is registry.redhat.io/devspaces/udi-rhel8@sha256:b5bce679581d79023180ed446f5fa6307639cf73202725bb4fda18c488dfc557' (quay.io/devspaces/udi-rhel8:3.5-3.5-16) in python-sample:

python-sample DevWorkspace.yaml

apiVersion: workspace.devfile.io/v1alpha2
metadata:
  annotations:
    che.eclipse.org/che-editor: che-incubator/che-code/latest
    che.eclipse.org/devfile-source: |
      factory:
        params: >-
          devWorkspace=https://devspaces.apps.ocp412.crw-qe.com/devfile-registry/devfiles/python__python-hello-world/devworkspace-che-code-latest.yaml&storageType=per-user&url=https://github.com/devspaces-samples/python-hello-world/tree/devspaces-3.5-rhel-8
    che.eclipse.org/last-updated-timestamp: '2023-03-16T12:26:42.998Z'
    controller.devfile.io/started-at: '1678969578398'
  creationTimestamp: '2023-03-16T12:26:02Z'
  finalizers:
    - storage.controller.devfile.io
    - rbac.controller.devfile.io
  generation: 4
  labels:
    controller.devfile.io/creator: 96f6cc27-5cd0-4fcd-917c-198d9e50f91c
  managedFields:
    - apiVersion: workspace.devfile.io/v1alpha2
      fieldsType: FieldsV1
      fieldsV1:
        'f:metadata':
          'f:annotations':
            .: {}
            'f:che.eclipse.org/che-editor': {}
            'f:che.eclipse.org/devfile-source': {}
            'f:che.eclipse.org/last-updated-timestamp': {}
        'f:spec':
          .: {}
          'f:contributions': {}
          'f:routingClass': {}
          'f:started': {}
          'f:template':
            .: {}
            'f:attributes':
              .: {}
              'f:controller.devfile.io/devworkspace-config':
                .: {}
                'f:name': {}
                'f:namespace': {}
              'f:controller.devfile.io/storage-type': {}
            'f:commands': {}
            'f:components': {}
            'f:projects': {}
      manager: unknown
      operation: Update
      time: '2023-03-16T12:26:06Z'
    - apiVersion: workspace.devfile.io/v1alpha2
      fieldsType: FieldsV1
      fieldsV1:
        'f:metadata':
          'f:annotations':
            'f:controller.devfile.io/started-at': {}
        'f:status':
          'f:conditions': {}
          'f:mainUrl': {}
          'f:message': {}
          'f:phase': {}
      manager: devworkspace-controller
      operation: Update
      time: '2023-03-16T12:26:18Z'
  name: python-hello-world
  namespace: admin-devspaces
  resourceVersion: '12134813'
  uid: 9f0ad4c2-621b-4bdc-a7e3-519736d92664
kind: DevWorkspace
spec:
  contributions:
    - kubernetes:
        name: che-code-python-hello-world
      name: editor
  routingClass: che
  started: true
  template:
    attributes:
      controller.devfile.io/devworkspace-config:
        name: devworkspace-config
        namespace: openshift-devspaces
      controller.devfile.io/storage-type: per-user
    projects:
      - name: python-hello-world
        zip:
          location: 'http://devfile-registry.openshift-devspaces.svc:8080/resources/v2/python-hello-world.zip'
    components:
      - attributes:
          controller.devfile.io/merge-contribution: true
        container:
          env:
            - name: CHE_DASHBOARD_URL
              value: 'https://devspaces.apps.ocp412.crw-qe.com'
            - name: CHE_PLUGIN_REGISTRY_URL
              value: 'https://devspaces.apps.ocp412.crw-qe.com/plugin-registry/v3'
            - name: CHE_PLUGIN_REGISTRY_INTERNAL_URL
              value: 'http://plugin-registry.openshift-devspaces.svc:8080/v3'
            - name: OPENVSX_REGISTRY_URL
              value: 'https://open-vsx.org'
          image: 'registry.redhat.io/devspaces/udi-rhel8@sha256:b5bce679581d79023180ed446f5fa6307639cf73202725bb4fda18c488dfc557'
          memoryLimit: 2G
          mountSources: true
          sourceMapping: /projects
          volumeMounts:
            - name: venv
              path: /home/user/.venv
        name: python
      - name: venv
        volume:
          size: 1G
    commands:
      - exec:
          commandLine: python -m venv .venv && . .venv/bin/activate && python hello-world.py
          component: python
          group:
            kind: run
          label: Run the application
          workingDir: '${PROJECTS_ROOT}/python-hello-world'
        id: run
status:
  conditions:
    - lastTransitionTime: '2023-03-16T12:26:06Z'
      message: DevWorkspace is starting
      status: 'True'
      type: Started
    - lastTransitionTime: '2023-03-16T12:26:06Z'
      message: Resolved plugins and parents from DevWorkspace
      status: 'True'
      type: DevWorkspaceResolved
    - lastTransitionTime: '2023-03-16T12:26:06Z'
      message: Storage ready
      status: 'True'
      type: StorageReady
    - lastTransitionTime: '2023-03-16T12:26:06Z'
      message: Networking ready
      status: 'True'
      type: RoutingReady
    - lastTransitionTime: '2023-03-16T12:26:06Z'
      message: DevWorkspace serviceaccount ready
      status: 'True'
      type: ServiceAccountReady
    - lastTransitionTime: '2023-03-16T12:26:06Z'
      message: DevWorkspace secrets ready
      status: 'True'
      type: PullSecretsReady
    - lastTransitionTime: '2023-03-16T12:26:14Z'
      message: DevWorkspace deployment ready
      status: 'True'
      type: DeploymentReady
    - lastTransitionTime: '2023-03-16T12:26:18Z'
      status: 'True'
      type: Ready
  devworkspaceId: workspace9f0ad4c2621b4bdc
  mainUrl: 'https://devspaces.apps.ocp412.crw-qe.com/workspace9f0ad4c2621b4bdc/python/3100/'
  message: 'https://devspaces.apps.ocp412.crw-qe.com/workspace9f0ad4c2621b4bdc/python/3100/'
  phase: Running

it is registry.redhat.io/devspaces/udi-rhel8@sha256:5d17d145844b4e6632aba478c7c66c6c5925cbb64ecf86b2c7083b2dfea6dfcb (quay.io/devspaces/udi-rhel8:3.5-16.16788811343.5-16) in empty workspace sample:

empty-workspace-sample DevWorkspace.yaml

apiVersion: workspace.devfile.io/v1alpha2
metadata:
  annotations:
    che.eclipse.org/devfile-source: |
      url:
        location: >-
          https://devspaces.apps.ocp412.crw-qe.com/dashboard/devfile-registry/devfiles/empty.yaml
      factory:
        params: >-
          storageType=per-user&url=https://devspaces.apps.ocp412.crw-qe.com/dashboard/devfile-registry/devfiles/empty.yaml
    che.eclipse.org/last-updated-timestamp: '2023-03-16T12:12:50.553Z'
    controller.devfile.io/started-at: '1678968760500'
  creationTimestamp: '2023-03-16T12:12:12Z'
  finalizers:
    - storage.controller.devfile.io
    - rbac.controller.devfile.io
  generation: 4
  labels:
    controller.devfile.io/creator: 96f6cc27-5cd0-4fcd-917c-198d9e50f91c
  managedFields:
    - apiVersion: workspace.devfile.io/v1alpha2
      fieldsType: FieldsV1
      fieldsV1:
        'f:metadata':
          'f:annotations':
            .: {}
            'f:che.eclipse.org/devfile-source': {}
            'f:che.eclipse.org/last-updated-timestamp': {}
        'f:spec':
          .: {}
          'f:contributions': {}
          'f:routingClass': {}
          'f:started': {}
          'f:template':
            .: {}
            'f:attributes':
              .: {}
              'f:controller.devfile.io/devworkspace-config':
                .: {}
                'f:name': {}
                'f:namespace': {}
              'f:controller.devfile.io/storage-type': {}
              'f:dw.metadata.annotations':
                .: {}
                'f:che.eclipse.org/devfile-source': {}
                'f:che.eclipse.org/last-updated-timestamp': {}
            'f:components': {}
      manager: unknown
      operation: Update
      time: '2023-03-16T12:12:13Z'
    - apiVersion: workspace.devfile.io/v1alpha2
      fieldsType: FieldsV1
      fieldsV1:
        'f:metadata':
          'f:annotations':
            'f:controller.devfile.io/started-at': {}
        'f:status':
          'f:conditions': {}
          'f:mainUrl': {}
          'f:message': {}
          'f:phase': {}
      manager: devworkspace-controller
      operation: Update
      time: '2023-03-16T12:12:40Z'
  name: empty-gslp
  namespace: admin-devspaces
  resourceVersion: '12127989'
  uid: b646262e-c211-49c9-b1cb-acc90393ad4b
kind: DevWorkspace
spec:
  contributions:
    - kubernetes:
        name: che-code-workspaceb646262ec21149c9
        namespace: admin-devspaces
      name: che-code-workspaceb646262ec21149c9
  routingClass: che
  started: true
  template:
    attributes:
      controller.devfile.io/devworkspace-config:
        name: devworkspace-config
        namespace: openshift-devspaces
      controller.devfile.io/storage-type: per-user
      dw.metadata.annotations:
        che.eclipse.org/devfile-source: |
          url:
            location: >-
              https://devspaces.apps.ocp412.crw-qe.com/dashboard/devfile-registry/devfiles/empty.yaml
          factory:
            params: >-
              storageType=per-user&url=https://devspaces.apps.ocp412.crw-qe.com/dashboard/devfile-registry/devfiles/empty.yaml
        che.eclipse.org/last-updated-timestamp: '2023-03-16T12:12:49.271Z'
    components:
      - attributes:
          che-code.eclipse.org/contribute-cpuLimit: true
          che-code.eclipse.org/contribute-cpuRequest: true
          che-code.eclipse.org/contribute-endpoint/che-code: 3100
          che-code.eclipse.org/contribute-endpoint/code-redirect-1: 13131
          che-code.eclipse.org/contribute-endpoint/code-redirect-2: 13132
          che-code.eclipse.org/contribute-endpoint/code-redirect-3: 13133
          che-code.eclipse.org/contribute-entry-point: true
          che-code.eclipse.org/contribute-memoryLimit: true
          che-code.eclipse.org/contribute-memoryRequest: true
          che-code.eclipse.org/contribute-volume-mount/checode: /checode
          che-code.eclipse.org/contributed-container: universal-developer-image
        container:
          command:
            - /checode/entrypoint-volume.sh
          cpuLimit: 500m
          cpuRequest: 30m
          endpoints:
            - attributes:
                contributed-by: che-code.eclipse.org
                cookiesAuthEnabled: true
                discoverable: false
                type: main
                urlRewriteSupported: true
              exposure: public
              name: che-code
              protocol: https
              secure: false
              targetPort: 3100
            - attributes:
                contributed-by: che-code.eclipse.org
                discoverable: false
                urlRewriteSupported: false
              exposure: public
              name: code-redirect-1
              protocol: http
              targetPort: 13131
            - attributes:
                contributed-by: che-code.eclipse.org
                discoverable: false
                urlRewriteSupported: false
              exposure: public
              name: code-redirect-2
              protocol: http
              targetPort: 13132
            - attributes:
                contributed-by: che-code.eclipse.org
                discoverable: false
                urlRewriteSupported: false
              exposure: public
              name: code-redirect-3
              protocol: http
              targetPort: 13133
          env:
            - name: CHE_DASHBOARD_URL
              value: 'https://devspaces.apps.ocp412.crw-qe.com'
            - name: CHE_PLUGIN_REGISTRY_URL
              value: 'https://devspaces.apps.ocp412.crw-qe.com/plugin-registry/v3'
            - name: CHE_PLUGIN_REGISTRY_INTERNAL_URL
              value: 'http://plugin-registry.openshift-devspaces.svc:8080/v3'
            - name: OPENVSX_REGISTRY_URL
              value: 'https://open-vsx.org'
          image: 'registry.redhat.io/devspaces/udi-rhel8@sha256:5d17d145844b4e6632aba478c7c66c6c5925cbb64ecf86b2c7083b2dfea6dfcb'
          memoryLimit: 1024Mi
          memoryRequest: 256Mi
          sourceMapping: /projects
          volumeMounts:
            - name: checode
              path: /checode
        name: universal-developer-image
status:
  conditions:
    - lastTransitionTime: '2023-03-16T12:12:13Z'
      message: DevWorkspace is starting
      status: 'True'
      type: Started
    - lastTransitionTime: '2023-03-16T12:12:13Z'
      message: Resolved plugins and parents from DevWorkspace
      status: 'True'
      type: DevWorkspaceResolved
    - lastTransitionTime: '2023-03-16T12:12:13Z'
      message: Storage ready
      status: 'True'
      type: StorageReady
    - lastTransitionTime: '2023-03-16T12:12:14Z'
      message: Networking ready
      status: 'True'
      type: RoutingReady
    - lastTransitionTime: '2023-03-16T12:12:14Z'
      message: DevWorkspace serviceaccount ready
      status: 'True'
      type: ServiceAccountReady
    - lastTransitionTime: '2023-03-16T12:12:14Z'
      message: DevWorkspace secrets ready
      status: 'True'
      type: PullSecretsReady
    - lastTransitionTime: '2023-03-16T12:12:36Z'
      message: DevWorkspace deployment ready
      status: 'True'
      type: DeploymentReady
    - lastTransitionTime: '2023-03-16T12:12:40Z'
      status: 'True'
      type: Ready
  devworkspaceId: workspaceb646262ec21149c9
  mainUrl: 'https://devspaces.apps.ocp412.crw-qe.com/workspaceb646262ec21149c9/universal-developer-image/3100/'
  message: 'https://devspaces.apps.ocp412.crw-qe.com/workspaceb646262ec21149c9/universal-developer-image/3100/'
  phase: Running

registry.stage.redhat.io/devspaces/udi-rhel8@sha256:5d17d145844b4e6632aba478c7c66c6c5925cbb64ecf86b2c7083b2dfea6dfcb can't be pulled

$ podman pull registry.stage.redhat.io/devspaces/udi-rhel8@sha256:5d17d145844b4e6632aba478c7c66c6c5925cbb64ecf86b2c7083b2dfea6dfcb
Trying to pull registry.stage.redhat.io/devspaces/udi-rhel8@sha256:5d17d145844b4e6632aba478c7c66c6c5925cbb64ecf86b2c7083b2dfea6dfcb...
Error: initializing source docker://registry.stage.redhat.io/devspaces/udi-rhel8@sha256:5d17d145844b4e6632aba478c7c66c6c5925cbb64ecf86b2c7083b2dfea6dfcb: reading manifest sha256:5d17d145844b4e6632aba478c7c66c6c5925cbb64ecf86b2c7083b2dfea6dfcb in registry.stage.redhat.io/devspaces/udi-rhel8: manifest unknown: manifest unknown

Possible root cause: unexpected CVE respins of the next images:

quay.io/devspaces/udi-rhel8:3.5-16.1678881134

registry.redhat.io/devspaces/udi-rhel8@sha256:5d17d145844b4e6632aba478c7c66c6c5925cbb64ecf86b2c7083b2dfea6dfcb is hard-coded in CheCluster:

    defaultComponents:
      - container:
          image: >-
            registry.redhat.io/devspaces/udi-rhel8@sha256:5d17d145844b4e6632aba478c7c66c6c5925cbb64ecf86b2c7083b2dfea6dfcb

Prerequisites (if any, like setup, operators/versions):

Steps to Reproduce

Deploy DS 3.5.0.RC-03-15 to OCP 4.10 using staging iib image:
Go to Dashboard and try to create Empty workspace

Actual results:

There was start workspace error

Failed to open the workspace
Container universal-developer-image has state ImagePullBackOff

Expected results:

Workspace has been started successfully.

Reproducibility (Always/Intermittent/Only Once):

Always

Build Details:

Additional info (Such as Logs, Screenshots, etc):

- - Sort By Name
  - Sort By Date
  - Ascending
  - Descending
  - Thumbnails
  - List
  - Download All

image-2023-03-16-12-11-50-439.png
111 kB
2023/03/16 3:11 PM
screenshot-1.png
1.37 MB
2023/03/16 2:47 PM
screenshot-2.png
150 kB
2023/03/16 2:48 PM
screenshot-3.png
256 kB
2023/03/16 2:59 PM
screenshot-4.png
99 kB
2023/03/16 10:53 PM
screenshot-5.png
304 kB
2023/03/16 10:54 PM
Screenshot from 2023-03-16 14-21-39.png
56 kB
2023/03/16 12:31 PM
Screenshot from 2023-03-16 14-33-35.png
187 kB
2023/03/16 12:33 PM
x86_64-ds_3.5.0_rc_03_16.png
74 kB
2023/03/17 10:26 AM

Details

Description