Uploaded image for project: 'Red Hat OpenShift Dev Spaces (formerly CodeReady Workspaces) '
  1. Red Hat OpenShift Dev Spaces (formerly CodeReady Workspaces)
  2. CRW-811

Workspaces created with CRW 2.0 fail to start after update to 2.1 when OpenShift OAuth support is enabled

XMLWordPrintable

      Existing workspaces that were created in an CRW 2.0.0 installation cannot be started again after the Che server is updated to CRW 2.1.0.

      Steps to reproduce

      1. workspace created in a 2.0.0 CRW installation with default settings (it will be created in a dedicated namespace named with the workspaceId)
      2. stop the workspace
      3. Upgrade CRW to 2.1.0 (either through OperatorHub or crwctl)
      4. Start existing workspace: it will produce this error.

      Expected behavior

      Workspace should start without error inside its original namespace

      Installation method

      Either with crwctl or OperatorHub

      che config map of the 2.0.0 Che server install with default settings + TLS (=> Openshift OAuth + TLS + selfsigned certs):

        CHE_INFRA_KUBERNETES_PVC_PRECREATE__SUBPATHS: 'true'
  CHE_PORT: '8080'
  CHE_INFRA_KUBERNETES_PVC_STRATEGY: per-workspace
  CHE_JDBC_PASSWORD: hDw1kyi6nke2
  CHE_KEYCLOAK_REALM: codeready
  CHE_INFRA_KUBERNETES_TRUST__CERTS: 'true'
  CHE_INFRASTRUCTURE_ACTIVE: openshift
  CHE_WORKSPACE_NO__PROXY: ''
  CHE_WORKSPACE_PLUGIN__BROKER_UNIFIED_IMAGE: 'registry.redhat.io/codeready-workspaces/pluginbroker-rhel8:2.0'
  CHE_INFRA_OPENSHIFT_PROJECT: ''
  CHE_MULTIUSER: 'true'
  CHE_HOST: codeready-test-david-prod-update.apps.test-ocp43.codereadyqe.com
  CHE_SERVER_SECURE__EXPOSER_JWTPROXY_IMAGE: 'registry.redhat.io/codeready-workspaces/jwtproxy-rhel8:2.0'
  CHE_WEBSOCKET_ENDPOINT__MINOR: >-
    wss://codeready-test-david-prod-update.apps.test-ocp43.codereadyqe.com/api/websocket-minor
  CHE_LOG_LEVEL: INFO
  CHE_JDBC_URL: 'jdbc:postgresql://postgres:5432/dbche'
  CHE_WORKSPACE_DEVFILE__REGISTRY__URL: >-
    https://devfile-registry-test-david-prod-update.apps.test-ocp43.codereadyqe.com
  CHE_WEBSOCKET_ENDPOINT: >-
    wss://codeready-test-david-prod-update.apps.test-ocp43.codereadyqe.com/api/websocket
  CHE_WORKSPACE_HTTPS__PROXY: ''
  CHE_DEBUG_SERVER: 'false'
  CHE_INFRA_KUBERNETES_SERVICE__ACCOUNT__NAME: che-workspace
  CHE_INFRA_OPENSHIFT_OAUTH__IDENTITY__PROVIDER: openshift-v4
  CHE_WORKSPACE_JAVA__OPTIONS: >-
    -XX:MaxRAM=150m -XX:MaxRAMFraction=2 -XX:+UseParallelGC
    -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4
    -XX:AdaptiveSizePolicyWeight=90 -Dsun.zip.disableMemoryMapping=true -Xms20m
    -Djava.security.egd=file:/dev/./urandom 
  CHE_KEYCLOAK_CLIENT__ID: codeready-public
  CHE_INFRA_KUBERNETES_PVC_STORAGE__CLASS__NAME: ''
  CHE_KEYCLOAK_AUTH__SERVER__URL: 'https://keycloak-test-david-prod-update.apps.test-ocp43.codereadyqe.com/auth'
  CHE_WORKSPACE_PLUGIN__REGISTRY__URL: >-
    https://plugin-registry-test-david-prod-update.apps.test-ocp43.codereadyqe.com/v3
  CHE_WORKSPACE_PLUGIN__BROKER_INIT_IMAGE: 'registry.redhat.io/codeready-workspaces/pluginbrokerinit-rhel8:2.0'
  CHE_INFRA_KUBERNETES_PVC_QUANTITY: 1Gi
  CHE_JDBC_USERNAME: pgche
  CHE_INFRA_OPENSHIFT_TLS__ENABLED: 'true'
  KUBERNETES_LABEL: 'app=codeready,component=codeready'
  CHE_API: 'https://codeready-test-david-prod-update.apps.test-ocp43.codereadyqe.com/api'
  CHE_WORKSPACE_MAVEN__OPTIONS: >-
    -XX:MaxRAM=150m -XX:MaxRAMFraction=2 -XX:+UseParallelGC
    -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4
    -XX:AdaptiveSizePolicyWeight=90 -Dsun.zip.disableMemoryMapping=true -Xms20m
    -Djava.security.egd=file:/dev/./urandom 
  JAVA_OPTS: >-
    -XX:MaxRAMFraction=2 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10
    -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90
    -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap
    -Dsun.zip.disableMemoryMapping=true -Xms20m 
  CHE_WORKSPACE_HTTP__PROXY: ''
  CHE_INFRA_KUBERNETES_PVC_JOBS_IMAGE: 'registry.redhat.io/ubi8-minimal:8.0-213'
  CHE_WORKSPACE_HTTP__PROXY__JAVA__OPTIONS: ''

      Che server POD env variables

          env:
        - name: CM_REVISION
          value: '4727172'
        - name: KUBERNETES_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        - name: CHE_SELF__SIGNED__CERT
          valueFrom:
            secretKeyRef:
              name: self-signed-certificate
              key: ca.crt
              optional: true

      Now Config map of the 2.1.0 Che server:

        CHE_INFRA_KUBERNETES_PVC_PRECREATE__SUBPATHS: 'true'
  CHE_PORT: '8080'
  CHE_INFRA_KUBERNETES_PVC_STRATEGY: per-workspace
  CHE_JDBC_PASSWORD: hDw1kyi6nke2
  CHE_KEYCLOAK_REALM: codeready
  CHE_INFRA_KUBERNETES_TRUST__CERTS: 'true'
  CHE_INFRASTRUCTURE_ACTIVE: openshift
  CHE_WORKSPACE_NO__PROXY: ''
  CHE_INFRA_KUBERNETES_NAMESPACE_ALLOW__USER__DEFINED: 'false'
  CHE_MULTIUSER: 'true'
  CHE_METRICS_ENABLED: 'false'
  CHE_HOST: codeready-test-david-prod-update.apps.test-ocp43.codereadyqe.com
  CHE_SERVER_SECURE__EXPOSER_JWTPROXY_IMAGE: >-
    registry.redhat.io/codeready-workspaces/jwtproxy-rhel8@sha256:63182dae6377ac01fbebdcb9c6a4435681ea88521cfc6e4222fe3920a3641127
  CHE_WEBSOCKET_ENDPOINT__MINOR: >-
    wss://codeready-test-david-prod-update.apps.test-ocp43.codereadyqe.com/api/websocket-minor
  CHE_LOG_LEVEL: INFO
  CHE_JDBC_URL: 'jdbc:postgresql://postgres:5432/dbche'
  CHE_WORKSPACE_DEVFILE__REGISTRY__URL: >-
    https://devfile-registry-test-david-prod-update.apps.test-ocp43.codereadyqe.com
  CHE_WEBSOCKET_ENDPOINT: >-
    wss://codeready-test-david-prod-update.apps.test-ocp43.codereadyqe.com/api/websocket
  CHE_WORKSPACE_HTTPS__PROXY: ''
  CHE_DEBUG_SERVER: 'false'
  CHE_INFRA_KUBERNETES_SERVICE__ACCOUNT__NAME: che-workspace
  CHE_INFRA_OPENSHIFT_OAUTH__IDENTITY__PROVIDER: openshift-v4
  CHE_INFRA_KUBERNETES_NAMESPACE_DEFAULT: <username>-codeready
  CHE_WORKSPACE_JAVA__OPTIONS: >-
    -XX:MaxRAM=150m -XX:MaxRAMFraction=2 -XX:+UseParallelGC
    -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4
    -XX:AdaptiveSizePolicyWeight=90 -Dsun.zip.disableMemoryMapping=true -Xms20m
    -Djava.security.egd=file:/dev/./urandom 
  CHE_WORKSPACE_PLUGIN__BROKER_ARTIFACTS_IMAGE: >-
    registry.redhat.io/codeready-workspaces/pluginbroker-artifacts-rhel8@sha256:5815bab69fc343cbf6dac0fd67dd70a25757fac08689a15e4a762655fa2e8a2c
  CHE_KEYCLOAK_CLIENT__ID: codeready-public
  CHE_WORKSPACE_PLUGIN__BROKER_METADATA_IMAGE: >-
    registry.redhat.io/codeready-workspaces/pluginbroker-metadata-rhel8@sha256:6c9abe63a70a6146dc49845f2f7732e3e6e0bcae6a19c3a6557367d6965bc1f8
  CHE_INFRA_KUBERNETES_PVC_STORAGE__CLASS__NAME: ''
  CHE_KEYCLOAK_AUTH__SERVER__URL: 'https://keycloak-test-david-prod-update.apps.test-ocp43.codereadyqe.com/auth'
  CHE_WORKSPACE_PLUGIN__REGISTRY__URL: >-
    https://plugin-registry-test-david-prod-update.apps.test-ocp43.codereadyqe.com/v3
  CHE_INFRA_KUBERNETES_PVC_QUANTITY: 1Gi
  KUBERNETES_LABELS: 'app=codeready,component=codeready'
  CHE_JDBC_USERNAME: pgche
  CHE_INFRA_OPENSHIFT_TLS__ENABLED: 'true'
  CHE_API: 'https://codeready-test-david-prod-update.apps.test-ocp43.codereadyqe.com/api'
  CHE_WORKSPACE_MAVEN__OPTIONS: >-
    -XX:MaxRAM=150m -XX:MaxRAMFraction=2 -XX:+UseParallelGC
    -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4
    -XX:AdaptiveSizePolicyWeight=90 -Dsun.zip.disableMemoryMapping=true -Xms20m
    -Djava.security.egd=file:/dev/./urandom 
  JAVA_OPTS: >-
    -XX:MaxRAMFraction=2 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10
    -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90
    -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap
    -Dsun.zip.disableMemoryMapping=true -Xms20m 
  CHE_WORKSPACE_HTTP__PROXY: ''
  CHE_INFRA_KUBERNETES_PVC_JOBS_IMAGE: >-
    registry.access.redhat.com/ubi8-minimal@sha256:9285da611437622492f9ef4229877efe302589f1401bbd4052e9bb261b3d4387
  CHE_WORKSPACE_HTTP__PROXY__JAVA__OPTIONS: ''

      And additional variables of the new 2.1.0 Che server:

              - name: CM_REVISION
          value: '4732509'
        - name: KUBERNETES_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        - name: CHE_SELF__SIGNED__CERT
          valueFrom:
            secretKeyRef:
              name: self-signed-certificate
              key: ca.crt
              optional: true
        - name: CHE_GIT_SELF__SIGNED__CERT
        - name: CHE_GIT_SELF__SIGNED__CERT__HOST

      Che server logs

      15-Apr-2020 12:51:47.844 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 13244 ms
2020-04-15 12:53:53,027[nio-8080-exec-4]  [ERROR] [o.e.c.a.w.s.WorkspaceRuntimes 494]   - Failure executing: PUT at: https://172.30.0.1/apis/project.openshift.io/v1/projects/workspace0fyulpv8u0w7ne3s. Message: Project.project.openshift.io "workspace0fyulpv8u0w7ne3s" is invalid: metadata.labels[che-managed]: Invalid value: "true": field is immutable, , try updating the namespace. Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=metadata.labels[che-managed], message=Invalid value: "true": field is immutable, , try updating the namespace, reason=FieldValueInvalid, additionalProperties={})], group=project.openshift.io, kind=Project, name=workspace0fyulpv8u0w7ne3s, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Project.project.openshift.io "workspace0fyulpv8u0w7ne3s" is invalid: metadata.labels[che-managed]: Invalid value: "true": field is immutable, , try updating the namespace, metadata=ListMeta(_continue=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).
org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInfrastructureException: Failure executing: PUT at: https://172.30.0.1/apis/project.openshift.io/v1/projects/workspace0fyulpv8u0w7ne3s. Message: Project.project.openshift.io "workspace0fyulpv8u0w7ne3s" is invalid: metadata.labels[che-managed]: Invalid value: "true": field is immutable, , try updating the namespace. Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=metadata.labels[che-managed], message=Invalid value: "true": field is immutable, , try updating the namespace, reason=FieldValueInvalid, additionalProperties={})], group=project.openshift.io, kind=Project, name=workspace0fyulpv8u0w7ne3s, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Project.project.openshift.io "workspace0fyulpv8u0w7ne3s" is invalid: metadata.labels[che-managed]: Invalid value: "true": field is immutable, , try updating the namespace, metadata=ListMeta(_continue=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).
	at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.update(OpenShiftProject.java:193)
	at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.prepare(OpenShiftProject.java:120)
	at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProjectFactory.getOrCreate(OpenShiftProjectFactory.java:86)
	at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftRuntimeContext.getRuntime(OpenShiftRuntimeContext.java:69)
	at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftRuntimeContext.getRuntime(OpenShiftRuntimeContext.java:31)
	at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.startAsync(WorkspaceRuntimes.java:466)
	at org.eclipse.che.api.workspace.server.WorkspaceManager.startAsync(WorkspaceManager.java:488)
	at org.eclipse.che.api.workspace.server.WorkspaceManager.startWorkspace(WorkspaceManager.java:373)
	at org.eclipse.che.multiuser.resource.api.workspace.LimitsCheckingWorkspaceManager.startWorkspace(LimitsCheckingWorkspaceManager.java:132)
	at org.eclipse.che.api.workspace.server.WorkspaceService.startById(WorkspaceService.java:458)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.everrest.core.impl.method.DefaultMethodInvoker.invokeMethod(DefaultMethodInvoker.java:140)
	at org.everrest.core.impl.method.DefaultMethodInvoker.invokeMethod(DefaultMethodInvoker.java:60)
	at org.everrest.core.impl.RequestDispatcher.doInvokeResource(RequestDispatcher.java:306)
	at org.everrest.core.impl.RequestDispatcher.invokeSubResourceMethod(RequestDispatcher.java:297)
	at org.everrest.core.impl.RequestDispatcher.dispatch(RequestDispatcher.java:233)
	at org.everrest.core.impl.RequestDispatcher.dispatch(RequestDispatcher.java:128)
	at org.everrest.core.impl.RequestHandlerImpl.handleRequest(RequestHandlerImpl.java:62)
	at org.everrest.core.impl.EverrestProcessor.process(EverrestProcessor.java:120)
	at org.everrest.core.servlet.EverrestServlet.service(EverrestServlet.java:61)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
	at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:290)
	at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:280)
	at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:184)
	at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:89)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
	at org.eclipse.che.commons.logback.filter.IdentityIdLoggerFilter.doFilter(IdentityIdLoggerFilter.java:49)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at org.eclipse.che.multiuser.api.authentication.commons.filter.MultiUserEnvironmentInitializationFilter.doFilter(MultiUserEnvironmentInitializationFilter.java:142)
	at org.eclipse.che.multiuser.keycloak.server.KeycloakEnvironmentInitializationFilter.doFilter(KeycloakEnvironmentInitializationFilter.java:88)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at org.eclipse.che.multiuser.machine.authentication.server.MachineLoginFilter.doFilter(MachineLoginFilter.java:76)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at org.eclipse.che.commons.logback.filter.RequestIdLoggerFilter.doFilter(RequestIdLoggerFilter.java:50)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:121)
	at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:133)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:528)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
	at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:798)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:810)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: PUT at: https://172.30.0.1/apis/project.openshift.io/v1/projects/workspace0fyulpv8u0w7ne3s. Message: Project.project.openshift.io "workspace0fyulpv8u0w7ne3s" is invalid: metadata.labels[che-managed]: Invalid value: "true": field is immutable, , try updating the namespace. Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=metadata.labels[che-managed], message=Invalid value: "true": field is immutable, , try updating the namespace, reason=FieldValueInvalid, additionalProperties={})], group=project.openshift.io, kind=Project, name=workspace0fyulpv8u0w7ne3s, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Project.project.openshift.io "workspace0fyulpv8u0w7ne3s" is invalid: metadata.labels[che-managed]: Invalid value: "true": field is immutable, , try updating the namespace, metadata=ListMeta(_continue=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:476)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:415)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:381)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:344)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleReplace(OperationSupport.java:261)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleReplace(OperationSupport.java:243)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleReplace(BaseOperation.java:785)
	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation$2.apply(HasMetadataOperation.java:109)
	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation$2.apply(HasMetadataOperation.java:104)
	at io.fabric8.openshift.api.model.DoneableProject.done(DoneableProject.java:27)
	at io.fabric8.openshift.api.model.DoneableProject.done(DoneableProject.java:6)
	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:116)
	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:37)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:420)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:415)
	at org.eclipse.che.workspace.infrastructure.openshift.project.OpenShiftProject.update(OpenShiftProject.java:186)
	... 58 common frames omitted

        1. image-2020-04-15-16-15-01-499.png
          image-2020-04-15-16-15-01-499.png
          17 kB

              skabashn Sergii Kabashniuk
              dfestal David Festal
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: