Uploaded image for project: 'Maistra'
  1. Maistra
  2. MAISTRA-927

Ingress and Egress Gateway Don't start in Maistra 1.0.0

    Details

      Description

      oc logs -n istio-system istio-ingressgateway-fd6c967b5-tl4lj
      2019-09-09T18:37:35.401457Z info FLAG: --applicationPorts="[]"
      2019-09-09T18:37:35.401510Z info FLAG: --binaryPath="/usr/local/bin/envoy"
      2019-09-09T18:37:35.401521Z info FLAG: --concurrency="0"
      2019-09-09T18:37:35.401532Z info FLAG: --configPath="/etc/istio/proxy"
      2019-09-09T18:37:35.401540Z info FLAG: --connectTimeout="10s"
      2019-09-09T18:37:35.401546Z info FLAG: --controlPlaneAuthPolicy="NONE"
      2019-09-09T18:37:35.401562Z info FLAG: --controlPlaneBootstrap="true"
      2019-09-09T18:37:35.401568Z info FLAG: --customConfigFile=""
      2019-09-09T18:37:35.401574Z info FLAG: --datadogAgentAddress=""
      2019-09-09T18:37:35.401580Z info FLAG: --disableInternalTelemetry="false"
      2019-09-09T18:37:35.401587Z info FLAG: --discoveryAddress="istio-pilot:15010"
      2019-09-09T18:37:35.401594Z info FLAG: --domain="istio-system.svc.cluster.local"
      2019-09-09T18:37:35.401600Z info FLAG: --drainDuration="45s"
      2019-09-09T18:37:35.401606Z info FLAG: --envoyMetricsServiceAddress=""
      2019-09-09T18:37:35.401611Z info FLAG: --help="false"
      2019-09-09T18:37:35.401617Z info FLAG: --id=""
      2019-09-09T18:37:35.401622Z info FLAG: --ip=""
      2019-09-09T18:37:35.401628Z info FLAG: --lightstepAccessToken=""
      2019-09-09T18:37:35.401637Z info FLAG: --lightstepAddress=""
      2019-09-09T18:37:35.401643Z info FLAG: --lightstepCacertPath=""
      2019-09-09T18:37:35.401648Z info FLAG: --lightstepSecure="false"
      2019-09-09T18:37:35.401654Z info FLAG: --log_as_json="false"
      2019-09-09T18:37:35.401660Z info FLAG: --log_caller=""
      2019-09-09T18:37:35.401665Z info FLAG: --log_output_level="default:info"
      2019-09-09T18:37:35.401671Z info FLAG: --log_rotate=""
      2019-09-09T18:37:35.401677Z info FLAG: --log_rotate_max_age="30"
      2019-09-09T18:37:35.401683Z info FLAG: --log_rotate_max_backups="1000"
      2019-09-09T18:37:35.401689Z info FLAG: --log_rotate_max_size="104857600"
      2019-09-09T18:37:35.401694Z info FLAG: --log_stacktrace_level="default:none"
      2019-09-09T18:37:35.401708Z info FLAG: --log_target="[stdout]"
      2019-09-09T18:37:35.401715Z info FLAG: --parentShutdownDuration="1m0s"
      2019-09-09T18:37:35.401724Z info FLAG: --proxyAdminPort="15000"
      2019-09-09T18:37:35.401730Z info FLAG: --proxyLogLevel="warning"
      2019-09-09T18:37:35.401736Z info FLAG: --serviceCluster="istio-ingressgateway"
      2019-09-09T18:37:35.401742Z info FLAG: --serviceregistry="Kubernetes"
      2019-09-09T18:37:35.401748Z info FLAG: --statsdUdpAddress=""
      2019-09-09T18:37:35.401755Z info FLAG: --statusPort="15020"
      2019-09-09T18:37:35.401760Z info FLAG: --templateFile=""
      2019-09-09T18:37:35.401766Z info FLAG: --trust-domain=""
      2019-09-09T18:37:35.401773Z info FLAG: --zipkinAddress="zipkin:9411"
      2019-09-09T18:37:35.401792Z info Version redhat@redhat-docker.io/maistra-1.0.0-c7ec7d4dfbe97812802f21c7cdb843bf62bed1a2-Clean
      2019-09-09T18:37:35.402252Z info Obtained private IP [10.128.2.18 fe80::9b:e9ff:fece:b04a]
      2019-09-09T18:37:35.402323Z info Proxy role: &model.Proxy{ClusterID:"", Type:"router", IPAddresses:[]string

      {"10.128.2.18", "10.128.2.18", "fe80::9b:e9ff:fece:b04a"}

      , ID:"istio-ingressgateway-fd6c967b5-tl4lj.istio-system", Locality*core.Locality)(nil), DNSDomain:"istio-system.svc.cluster.local", ConfigNamespace:"", TrustDomain:"cluster.local", Metadata:map[string]string{}, SidecarScope*model.SidecarScope)(nil), ServiceInstances:[]*model.ServiceInstance(nil), WorkloadLabels:model.LabelsCollection(nil)}
      2019-09-09T18:37:35.402340Z info PilotSAN []string(nil)
      2019-09-09T18:37:35.402912Z info Effective config: binaryPath: /usr/local/bin/envoy
      configPath: /etc/istio/proxy
      connectTimeout: 10s
      discoveryAddress: istio-pilot:15010
      drainDuration: 45s
      parentShutdownDuration: 60s
      proxyAdminPort: 15000
      serviceCluster: istio-ingressgateway
      statNameLength: 189
      tracing:
      zipkin:
      address: zipkin:9411

      2019-09-09T18:37:35.403088Z info Monitored certs: []string

      {"/etc/certs/cert-chain.pem", "/etc/certs/key.pem", "/etc/certs/root-cert.pem"}

      2019-09-09T18:37:35.403139Z info PilotSAN []string(nil)
      2019-09-09T18:37:35.403261Z info Starting proxy agent
      2019-09-09T18:37:35.403291Z info Opening status port 15020

      2019-09-09T18:37:35.403743Z info Received new config, resetting budget
      2019-09-09T18:37:35.403756Z info Reconciling retry (budget 10)
      2019-09-09T18:37:35.403774Z info Epoch 0 starting
      2019-09-09T18:37:35.403896Z error Failed to generate bootstrap config: open /var/lib/istio/envoy/envoy_bootstrap_tmpl.json: permission denied
      ➜ ~
      ➜ ~
      ➜ ~
      ➜ ~ oc get pods -n istio-system
      NAME READY STATUS RESTARTS AGE
      istio-citadel-66c68799dd-n6gcc 1/1 Running 0 26m
      istio-egressgateway-64c6b84f86-4wgqn 0/1 CrashLoopBackOff 7 12m
      istio-galley-6b5f7cb4c7-d7cwp 1/1 Running 0 14m
      istio-ingressgateway-fd6c967b5-tl4lj 0/1 CrashLoopBackOff 7 12m
      istio-pilot-cdf95747b-l262p 2/2 Running 0 13m
      istio-policy-5594b6b96f-g244m 2/2 Running 0 13m
      istio-telemetry-55c48594d7-gm26j 2/2 Running 0 13m
      jaeger-644d4c59dc-c795v 2/2 Running 0 13m
      prometheus-5c88cddd6-k2vm2 2/2 Running 0 14m
      ➜ ~
      ➜ ~
      ➜ ~ oc describe pod istio-ingressgateway-fd6c967b5-tl4lj
      Name: istio-ingressgateway-fd6c967b5-tl4lj
      Namespace: istio-system
      Priority: 0
      PriorityClassName: <none>
      Node: ip-10-0-148-137.ec2.internal/10.0.148.137
      Start Time: Mon, 09 Sep 2019 14:26:40 -0400
      Labels: app=istio-ingressgateway
      chart=gateways
      heritage=Tiller
      istio=ingressgateway
      maistra-control-plane=istio-system
      maistra-version=1.0.0
      pod-template-hash=fd6c967b5
      release=istio
      Annotations: k8s.v1.cni.cncf.io/networks-status:
      [{
      "name": "openshift-sdn",
      "interface": "eth0",
      "ips": [
      "10.128.2.18"
      ],
      "default": true,
      "dns": {}
      }]
      openshift.io/scc: restricted
      sidecar.istio.io/inject: false
      Status: Running
      IP: 10.128.2.18
      Controlled By: ReplicaSet/istio-ingressgateway-fd6c967b5
      Containers:
      istio-proxy:
      Container ID: cri-o://f806dfe60217c628dda5f6e657f34641d3eeff8b6571a65a89559cd93cae042d
      Image: docker.io/maistra/proxyv2-ubi8:1.0.0
      Image ID: docker.io/maistra/proxyv2-ubi8@sha256:3cd12d5da4e1a0d356c11239403cfb714c7ba3d645b76068d5162af95b7ac4cb
      Ports: 15020/TCP, 8080/TCP, 8443/TCP, 15443/TCP, 15090/TCP
      Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP
      Args:
      proxy
      router
      --domain
      $(POD_NAMESPACE).svc.cluster.local
      --log_output_level=default:info
      --drainDuration
      45s
      --parentShutdownDuration
      1m0s
      --connectTimeout
      10s
      --serviceCluster
      istio-ingressgateway
      --zipkinAddress
      zipkin:9411
      --proxyAdminPort
      15000
      --statusPort
      15020
      --controlPlaneAuthPolicy
      NONE
      --discoveryAddress
      istio-pilot:15010
      State: Waiting
      Reason: CrashLoopBackOff
      Last State: Terminated
      Reason: Error
      Exit Code: 1
      Started: Mon, 09 Sep 2019 14:37:35 -0400
      Finished: Mon, 09 Sep 2019 14:37:35 -0400
      Ready: False
      Restart Count: 7
      Limits:
      cpu: 2
      memory: 1Gi
      Requests:
      cpu: 100m
      memory: 128Mi
      Readiness: http-get http://:15020/healthz/ready delay=1s timeout=1s period=2s #success=1 #failure=30
      Environment:
      POD_NAME: istio-ingressgateway-fd6c967b5-tl4lj (v1:metadata.name)
      POD_NAMESPACE: istio-system (v1:metadata.namespace)
      INSTANCE_IP: (v1:status.podIP)
      HOST_IP: (v1:status.hostIP)
      ISTIO_META_POD_NAME: istio-ingressgateway-fd6c967b5-tl4lj (v1:metadata.name)
      ISTIO_META_CONFIG_NAMESPACE: istio-system (v1:metadata.namespace)
      ISTIO_META_ROUTER_MODE: sni-dnat
      Mounts:
      /etc/certs from istio-certs (ro)
      /etc/istio/ingressgateway-ca-certs from ingressgateway-ca-certs (ro)
      /etc/istio/ingressgateway-certs from ingressgateway-certs (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from istio-ingressgateway-service-account-token-vbnrd (ro)
      Conditions:
      Type Status
      Initialized True
      Ready False
      ContainersReady False
      PodScheduled True
      Volumes:
      istio-certs:
      Type: Secret (a volume populated by a Secret)
      SecretName: istio.istio-ingressgateway-service-account
      Optional: true
      ingressgateway-certs:
      Type: Secret (a volume populated by a Secret)
      SecretName: istio-ingressgateway-certs
      Optional: true
      ingressgateway-ca-certs:
      Type: Secret (a volume populated by a Secret)
      SecretName: istio-ingressgateway-ca-certs
      Optional: true
      istio-ingressgateway-service-account-token-vbnrd:
      Type: Secret (a volume populated by a Secret)
      SecretName: istio-ingressgateway-service-account-token-vbnrd
      Optional: false
      QoS Class: Burstable
      Node-Selectors: <none>
      Tolerations: node.kubernetes.io/memory-pressure:NoSchedule
      node.kubernetes.io/not-ready:NoExecute for 300s
      node.kubernetes.io/unreachable:NoExecute for 300s
      Events:
      Type Reason Age From Message
      ---- ------ ---- ---- -------
      Normal Scheduled 12m default-scheduler Successfully assigned istio-system/istio-ingressgateway-fd6c967b5-tl4lj to ip-10-0-148-137.ec2.internal
      Normal Pulled 11m (x5 over 12m) kubelet, ip-10-0-148-137.ec2.internal Container image "docker.io/maistra/proxyv2-ubi8:1.0.0" already present on machine
      Normal Created 11m (x5 over 12m) kubelet, ip-10-0-148-137.ec2.internal Created container istio-proxy
      Normal Started 11m (x5 over 12m) kubelet, ip-10-0-148-137.ec2.internal Started container istio-proxy
      Warning BackOff 2m29s (x49 over 12m) kubelet, ip-10-0-148-137.ec2.internal Back-off restarting failed container

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                kconner Kevin Conner
                Reporter:
                brian.avery Brian Avery
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: