Uploaded image for project: 'Red Hat Fuse'
  1. Red Hat Fuse
  2. ENTESB-8578

f8-arq hangs on "Waiting for container" when Completed pods in namespace

    Details

    • Sprint:
      FIS 2.0 Sprint 6

      Description

      After deploying the spring-boot-camel arechetype (via fabric8:run), the following pods exist in the namespace:

      $ oc get pods
      NAME                        READY     STATUS      RESTARTS   AGE
      spring-boot-camel-4-dp34c   1/1       Running     0          9m
      spring-boot-camel-9-build   0/1       Completed   0          10m
      

      Running the integration tests on this env via

      mvn -Dfabric8.generator.from=docker-registry.usersys.redhat.com/fabric8/fis-java-openshift:2.0 -Dfabric8.mode=openshift -Dfabric8.build.strategy=s2i -Dnamespace.use.existing=maschmid failsafe:integration-test
      

      Hangs the tests with repeated "Waiting for container:sti-build."

      $ mvn -Dfabric8.generator.from=docker-registry.usersys.redhat.com/fabric8/fis-java-openshift:2.0 -Dfabric8.mode=openshift -Dfabric8.build.strategy=s2i -Dnamespace.use.existing=maschmid failsafe:integration-test
      [INFO] Scanning for projects...
      [INFO]                                                                         
      [INFO] ------------------------------------------------------------------------
      [INFO] Building Fabric8 :: Quickstarts :: Spring-Boot :: Camel 1.0-SNAPSHOT
      [INFO] ------------------------------------------------------------------------
      [INFO] 
      [INFO] --- maven-failsafe-plugin:2.18.1:integration-test (default-cli) @ spring-boot-camel ---
      [INFO] Failsafe report directory: /home/maschmid/devel/xpaas-qe/fis-2.0/test-fuse/tmp/spring-boot-camel9191518875825284104/spring-boot-camel/target/failsafe-reports
      
      -------------------------------------------------------
       T E S T S
      -------------------------------------------------------
      Running com.redhat.xpaas.qe.KubernetesIntegrationKT
      Initializing Session:0d85249f-8110-45e8-b791-656cd6ab687f
      Using Kubernetes at: https://api.devel.xpaas:8443/
      Creating kubernetes resources inside namespace: maschmid
      if you use OpenShift then type this switch namespaces:     oc project maschmid
      if you use kubernetes then type this to switch namespaces: kubectl namespace maschmid
      13:50:58.964 [main] INFO  o.h.validator.internal.util.Version - HV000001: Hibernate Validator 5.2.4.Final
      13:50:59.608 [main] WARN  i.f.arquillian.utils.Namespaces - failed to update namespace: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: PATCH at: https://api.devel.xpaas:8443/api/v1/namespaces/maschmid. Message: Forbidden! User xpaasqe/api-devel-xpaas:8443 doesn't have permission..
      io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: PATCH at: https://api.devel.xpaas:8443/api/v1/namespaces/maschmid. Message: Forbidden! User xpaasqe/api-devel-xpaas:8443 doesn't have permission..
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:314)
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:265)
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:236)
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:229)
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handlePatch(OperationSupport.java:220)
      	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handlePatch(BaseOperation.java:555)
      	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation$3.apply(HasMetadataOperation.java:134)
      	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation$3.apply(HasMetadataOperation.java:130)
      	at io.fabric8.kubernetes.api.model.DoneableNamespace.done(DoneableNamespace.java:26)
      	at io.fabric8.kubernetes.api.model.DoneableNamespace.done(DoneableNamespace.java:5)
      	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.patch(HasMetadataOperation.java:141)
      	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation$1.apply(HasMetadataOperation.java:51)
      	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation$1.apply(HasMetadataOperation.java:41)
      	at io.fabric8.kubernetes.api.model.DoneableNamespace.done(DoneableNamespace.java:26)
      	at io.fabric8.arquillian.utils.Namespaces.updateNamespaceStatus(Namespaces.java:67)
      	at io.fabric8.arquillian.kubernetes.SessionListener.start(SessionListener.java:102)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
      	at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)
      	at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:81)
      	at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:145)
      	at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:116)
      	at org.jboss.arquillian.core.impl.EventImpl.fire(EventImpl.java:67)
      	at io.fabric8.arquillian.kubernetes.SuiteListener.start(SuiteListener.java:44)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
      	at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)
      	at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:81)
      	at org.jboss.arquillian.test.impl.TestContextHandler.createSuiteContext(TestContextHandler.java:73)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
      	at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)
      	at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:145)
      	at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:116)
      	at org.jboss.arquillian.test.impl.EventTestRunnerAdaptor.beforeSuite(EventTestRunnerAdaptor.java:75)
      	at org.jboss.arquillian.junit.Arquillian.run(Arquillian.java:115)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
      	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
      	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
      	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
      Applying kubernetes configuration from: file:/home/maschmid/devel/xpaas-qe/fis-2.0/test-fuse/tmp/spring-boot-camel9191518875825284104/spring-boot-camel/target/classes/META-INF/fabric8/openshift.yml
      No local fabric8 docker registry found
      Applying service:spring-boot-camel
      13:51:01.172 [main] INFO  io.fabric8.kubernetes.api.Controller - Deleting Service: spring-boot-camel
      13:51:01.309 [main] INFO  io.fabric8.kubernetes.api.Controller - Creating a Service from 0d85249f-8110-45e8-b791-656cd6ab687f namespace maschmid name spring-boot-camel
      13:51:01.464 [main] INFO  io.fabric8.kubernetes.api.Controller - Created Service: Service(apiVersion=v1, kind=Service, metadata=ObjectMeta(annotations={fabric8.io/git-branch=fis-2.0, fabric8.io/git-commit=89c5c1c303f0b68799e049b4637abcb6b4a80fc2, fabric8.io/iconUrl=img/icons/camel.svg, prometheus.io/port=9779, prometheus.io/scrape=true}, creationTimestamp=2016-09-20T11:51:01Z, deletionGracePeriodSeconds=null, deletionTimestamp=null, generateName=null, generation=null, labels={group=com.redhat.xpaas.qe, project=spring-boot-camel, provider=fabric8, version=1.0-SNAPSHOT}, name=spring-boot-camel, namespace=maschmid, resourceVersion=1150188, selfLink=/api/v1/namespaces/maschmid/services/spring-boot-camel, uid=8088f92b-7f28-11e6-984f-fa163ed7c622, additionalProperties={}), spec=ServiceSpec(clusterIP=172.30.158.68, deprecatedPublicIPs=[], externalIPs=[], loadBalancerIP=null, portalIP=172.30.158.68, ports=[ServicePort(name=null, nodePort=32102, port=8080, protocol=TCP, targetPort=IntOrString(IntVal=8080, Kind=null, StrVal=null, additionalProperties={}), additionalProperties={})], selector={group=com.redhat.xpaas.qe, project=spring-boot-camel, provider=fabric8}, sessionAffinity=None, type=LoadBalancer, additionalProperties={}), status=ServiceStatus(loadBalancer=LoadBalancerStatus(ingress=[], additionalProperties={}), additionalProperties={}), additionalProperties={})
      Applying route for:spring-boot-camel
      Applying DeploymentConfig.
      13:51:01.813 [main] INFO  io.fabric8.kubernetes.api.Controller - Deleting DeploymentConfig: spring-boot-camel
      13:51:01.944 [main] INFO  io.fabric8.kubernetes.api.Controller - Creating a DeploymentConfig from 0d85249f-8110-45e8-b791-656cd6ab687f namespace maschmid name spring-boot-camel
      13:51:02.106 [main] INFO  io.fabric8.kubernetes.api.Controller - Created DeploymentConfig: DeploymentConfig(apiVersion=v1, kind=DeploymentConfig, metadata=ObjectMeta(annotations={fabric8.io/docs-url=dav:http://content-repository/sites/com.redhat.xpaas.qe/spring-boot-camel/1.0-SNAPSHOT, fabric8.io/git-branch=fis-2.0, fabric8.io/git-commit=89c5c1c303f0b68799e049b4637abcb6b4a80fc2, fabric8.io/iconUrl=img/icons/camel.svg, fabric8.io/metrics-path=dashboard/file/camel-routes.json/?var-project=spring-boot-camel&var-version=1.0-SNAPSHOT}, creationTimestamp=2016-09-20T11:51:01Z, deletionGracePeriodSeconds=null, deletionTimestamp=null, generateName=null, generation=1, labels={group=com.redhat.xpaas.qe, project=spring-boot-camel, provider=fabric8, version=1.0-SNAPSHOT}, name=spring-boot-camel, namespace=maschmid, resourceVersion=1150197, selfLink=/oapi/v1/namespaces/maschmid/deploymentconfigs/spring-boot-camel, uid=80ea720c-7f28-11e6-984f-fa163ed7c622, additionalProperties={}), spec=DeploymentConfigSpec(replicas=1, selector={group=com.redhat.xpaas.qe, project=spring-boot-camel, provider=fabric8}, strategy=DeploymentStrategy(annotations=null, customParams=null, labels=null, recreateParams=null, resources=ResourceRequirements(limits=null, requests=null, additionalProperties={}), rollingParams=RollingDeploymentStrategyParams(intervalSeconds=1, maxSurge=IntOrString(IntVal=null, Kind=null, StrVal=25%, additionalProperties={}), maxUnavailable=IntOrString(IntVal=null, Kind=null, StrVal=25%, additionalProperties={}), post=null, pre=null, timeoutSeconds=10800, updatePercent=null, updatePeriodSeconds=1, additionalProperties={}), type=Rolling, additionalProperties={}), template=PodTemplateSpec(metadata=ObjectMeta(annotations={fabric8.io/docs-url=dav:http://content-repository/sites/com.redhat.xpaas.qe/spring-boot-camel/1.0-SNAPSHOT, fabric8.io/git-branch=fis-2.0, fabric8.io/git-commit=89c5c1c303f0b68799e049b4637abcb6b4a80fc2, fabric8.io/iconUrl=img/icons/camel.svg, fabric8.io/metrics-path=dashboard/file/camel-routes.json/?var-project=spring-boot-camel&var-version=1.0-SNAPSHOT}, creationTimestamp=null, deletionGracePeriodSeconds=null, deletionTimestamp=null, generateName=null, generation=null, labels={group=com.redhat.xpaas.qe, project=spring-boot-camel, provider=fabric8, version=1.0-SNAPSHOT}, name=null, namespace=null, resourceVersion=null, selfLink=null, uid=null, additionalProperties={}), spec=PodSpec(activeDeadlineSeconds=null, containers=[Container(args=[], command=[], env=[EnvVar(name=JAVA_OPTIONS, value=-Djava.security.egd=file:/dev/./urandom, valueFrom=null, additionalProperties={}), EnvVar(name=KUBERNETES_NAMESPACE, value=null, valueFrom=EnvVarSource(configMapKeyRef=null, fieldRef=ObjectFieldSelector(apiVersion=v1, fieldPath=metadata.namespace, additionalProperties={}), secretKeyRef=null, additionalProperties={}), additionalProperties={})], image=qe/spring-boot-camel:snapshot-160920-105006-0640, imagePullPolicy=IfNotPresent, lifecycle=null, livenessProbe=Probe(exec=null, failureThreshold=3, httpGet=HTTPGetAction(host=null, httpHeaders=[], path=/health, port=IntOrString(IntVal=8081, Kind=null, StrVal=null, additionalProperties={}), scheme=HTTP, additionalProperties={}), initialDelaySeconds=180, periodSeconds=10, successThreshold=1, tcpSocket=null, timeoutSeconds=1, additionalProperties={}), name=spring-boot, ports=[ContainerPort(containerPort=8080, hostIP=null, hostPort=null, name=http, protocol=TCP, additionalProperties={}), ContainerPort(containerPort=9779, hostIP=null, hostPort=null, name=prometheus, protocol=TCP, additionalProperties={}), ContainerPort(containerPort=8778, hostIP=null, hostPort=null, name=jolokia, protocol=TCP, additionalProperties={})], readinessProbe=Probe(exec=null, failureThreshold=3, httpGet=HTTPGetAction(host=null, httpHeaders=[], path=/health, port=IntOrString(IntVal=8081, Kind=null, StrVal=null, additionalProperties={}), scheme=HTTP, additionalProperties={}), initialDelaySeconds=10, periodSeconds=10, successThreshold=1, tcpSocket=null, timeoutSeconds=1, additionalProperties={}), resources=ResourceRequirements(limits=null, requests=null, additionalProperties={}), securityContext=SecurityContext(capabilities=null, privileged=false, readOnlyRootFilesystem=null, runAsNonRoot=null, runAsUser=null, seLinuxOptions=null, additionalProperties={}), stdin=null, stdinOnce=null, terminationMessagePath=/dev/termination-log, tty=null, volumeMounts=[], workingDir=null, additionalProperties={})], dnsPolicy=ClusterFirst, host=null, hostIPC=null, hostNetwork=null, hostPID=null, imagePullSecrets=[], nodeName=null, nodeSelector=null, restartPolicy=Always, securityContext=PodSecurityContext(fsGroup=null, runAsNonRoot=null, runAsUser=null, seLinuxOptions=null, supplementalGroups=[], additionalProperties={}), serviceAccount=null, serviceAccountName=null, terminationGracePeriodSeconds=30, volumes=[], additionalProperties={}), additionalProperties={}), test=false, triggers=[DeploymentTriggerPolicy(imageChangeParams=null, type=ConfigChange, additionalProperties={}), DeploymentTriggerPolicy(imageChangeParams=DeploymentTriggerImageChangeParams(automatic=true, containerNames=[spring-boot], from=ObjectReference(apiVersion=null, fieldPath=null, kind=ImageStreamTag, name=spring-boot-camel:snapshot-160920-105006-0640, namespace=maschmid, resourceVersion=null, uid=null, additionalProperties={}), lastTriggeredImage=null, additionalProperties={}), type=ImageChange, additionalProperties={})], additionalProperties={}), status=DeploymentConfigStatus(details=null, latestVersion=null, observedGeneration=null, additionalProperties={}), additionalProperties={})
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      Waiting for container:sti-build.
      

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                maschmid Marek Schmidt
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: