Uploaded image for project: 'OpenShift Service Mesh'
  1. OpenShift Service Mesh
  2. OSSM-1894

False positive 'Ready' condition in ServiceMeshMemberRoll

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Undefined Undefined
    • OSSM 2.2.2
    • OSSM 2.2.2
    • Maistra
    • None
    • False
    • None
    • False
    • Hide
      1. create basic smcp
      2. immediately create default smmr
      3. run 'oc wait --for condition=Ready -n istio-system smmr/default --timeout 300s'

      Condition is met immediately but pods in smcp namespace are still starting.

      Show
      create basic smcp immediately create default smmr run 'oc wait --for condition=Ready -n istio-system smmr/default --timeout 300s' Condition is met immediately but pods in smcp namespace are still starting.

      In current 2.2.2 build (openshift-istio-operator-rhel8-container-2.2.2-8) the 'Ready' condition in ServiceMeshMemberRoll shows false positive.

      It's marked as Ready immediately after creation of SMCP when pods are still starting.

      Here is a test which caught this - https://github.com/maistra/maistra-test-tool/blob/maistra-2.2/pkg/ossm/smcp_install.go#L74

      Here is a test log which shows that there are only 2s between SMCP creation and smmr showing Ready:

      time="2022-08-16T14:51:15Z" level=info msg="Create SMCP v2.1 in istio-system"
time="2022-08-16T14:51:15Z" level=info msg="Running command oc new-project istio-system"
time="2022-08-16T14:51:16Z" level=info msg="Running command kubectl apply -n istio-system -f /tmp/kubeapply925113892.yaml"
time="2022-08-16T14:51:16Z" level=info msg="Command output: \nservicemeshcontrolplane.maistra.io/basic created"
time="2022-08-16T14:51:16Z" level=info msg="Running command kubectl apply -n istio-system -f /tmp/kubeapply798885171.yaml"
time="2022-08-16T14:51:17Z" level=info msg="Command output: \nservicemeshmemberroll.maistra.io/default created"
time="2022-08-16T14:51:17Z" level=info msg="Waiting for mesh installation to complete"
time="2022-08-16T14:51:17Z" level=info msg="Running command oc wait --for condition=Ready -n istio-system smmr/default --timeout 300s"
time="2022-08-16T14:51:18Z" level=info msg="Command output: \nservicemeshmemberroll.maistra.io/default condition met"
time="2022-08-16T14:51:18Z" level=info msg="Verify SMCP status and pods"
time="2022-08-16T14:51:18Z" level=info msg="Running command oc get -n istio-system smcp/basic -o wide"
time="2022-08-16T14:51:18Z" level=info msg="Command output: \nNAME    READY   STATUS            PROFILES   VERSION   AGE   IMAGE REGISTRY\nbasic           ResourceCreated                        2s    "
time="2022-08-16T14:51:18Z" level=error msg="SMCP not Ready"
    smcp_install.go:92: SMCP not Ready
time="2022-08-16T14:51:18Z" level=info msg="Running command oc get -n istio-system pods"
time="2022-08-16T14:51:18Z" level=info msg="Command output: \nNAME                            READY   STATUS              RESTARTS   AGE\nistiod-basic-7cff65cb5b-tsfcs   0/1     ContainerCreating   0          0s"

      and here is the same test in 2.2.1:

      === RUN   A1/smcp_test_install_2.1
time="2022-08-16T14:32:22Z" level=info msg="Create SMCP v2.1 in istio-system"
time="2022-08-16T14:32:22Z" level=info msg="Running command oc new-project istio-system"
time="2022-08-16T14:32:22Z" level=info msg="Running command kubectl apply -n istio-system -f /tmp/kubeapply826249470.yaml"
time="2022-08-16T14:32:22Z" level=info msg="Command output: \nservicemeshcontrolplane.maistra.io/basic created"
time="2022-08-16T14:32:22Z" level=info msg="Running command kubectl apply -n istio-system -f /tmp/kubeapply334301253.yaml"
time="2022-08-16T14:32:22Z" level=info msg="Command output: \nservicemeshmemberroll.maistra.io/default created"
time="2022-08-16T14:32:22Z" level=info msg="Waiting for mesh installation to complete"
time="2022-08-16T14:32:22Z" level=info msg="Running command oc wait --for condition=Ready -n istio-system smmr/default --timeout 300s"
time="2022-08-16T14:32:47Z" level=info msg="Command output: \nservicemeshmemberroll.maistra.io/default condition met"
time="2022-08-16T14:32:47Z" level=info msg="Verify SMCP status and pods"
time="2022-08-16T14:32:47Z" level=info msg="Running command oc get -n istio-system smcp/basic -o wide"
time="2022-08-16T14:32:47Z" level=info msg="Command output: \nNAME    READY   STATUS            PROFILES      VERSION   AGE   IMAGE REGISTRY\nbasic   10/10   ComponentsReady   [\"default\"]   2.1.4     25s   "
time="2022-08-16T14:32:47Z" level=info msg="Running command oc get -n istio-system pods"
time="2022-08-16T14:32:47Z" level=info msg="Command output: \nNAME                                    READY   STATUS    RESTARTS   AGE\ngrafana-64d68f8ddd-cbgxz                2/2     Running   0          13s\nistio-egressgateway-684b45584f-dxzl2    1/1     Running   0          14s\nistio-ingressgateway-5964cc866b-zt2hl   1/1     Running   0          14s\nistiod-basic-7cff65cb5b-n8cnn           1/1     Running   0          24s\njaeger-6dff5d7bc4-fj74n                 2/2     Running   0          12s\nprometheus-768bb56667-rqnr5             2/2     Running   0          20s\nwasm-cacher-basic-d9dbcc97b-6dwst       1/1     Running   0          6s"

       

      You can see that in 2.2.1 it takes 25s before smmr is Ready.

       

      Full operator log attached

        1. istio-operator-6c58884fc7-4ts9p-istio-operator-fail.log
          676 kB

              yuaxu@redhat.com Yuanlin Xu
              fbrychta@redhat.com Filip Brychta
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: