Uploaded image for project: 'Red Hat Advanced Cluster Management'
  1. Red Hat Advanced Cluster Management
  2. ACM-7064

Insights client OOM crashlooping at 3500+ Managed Clusters

XMLWordPrintable

    • False
    • None
    • False
    • 9/27: pending further investigation
    • No

      Description of problem:

      While scale testing ACM 2.9 ZTP of 3500+ SNOs the insights client pod is OOM crashlooping at the conclusion of the test

      # oc get po -n open-cluster-management insights-client-bcd84b656-jpvngNAME                              READY   STATUS             RESTARTS         AGEinsights-client-bcd84b656-jpvng   0/1     CrashLoopBackOff   31 (2m57s ago)   15h

      Describe Output:

      # oc get po -n open-cluster-management insights-client-bcd84b656-jpvng
      NAME                              READY   STATUS             RESTARTS         AGE
      insights-client-bcd84b656-jpvng   0/1     CrashLoopBackOff   31 (2m57s ago)   15h# oc describe po -n open-cluster-management insights-client-bcd84b656-jpvng
      Name:             insights-client-bcd84b656-jpvng
      Namespace:        open-cluster-management
      Priority:         0
      Service Account:  insights-client
      Node:             e27-h03-000-r650/fc00:1004::6
      Start Time:       Wed, 23 Aug 2023 03:04:08 +0000
      Labels:           name=insights-client
                        ocm-antiaffinity-selector=insightsclient
                        pod-template-hash=bcd84b656
      Annotations:      k8s.ovn.org/pod-networks:
                          {"default":{"ip_addresses":["fd01:0:0:2::6b/64"],"mac_address":"0a:58:8e:c3:80:15","gateway_ips":["fd01:0:0:2::1"],"ip_address":"fd01:0:0:...
                        k8s.v1.cni.cncf.io/network-status:
                          [{
                              "name": "ovn-kubernetes",
                              "interface": "eth0",
                              "ips": [
                                  "fd01:0:0:2::6b"
                              ],
                              "mac": "0a:58:8e:c3:80:15",
                              "default": true,
                              "dns": {}
                          }]
                        openshift.io/scc: restricted-v2
                        seccomp.security.alpha.kubernetes.io/pod: runtime/default
      Status:           Running
      IP:               fd01:0:0:2::6b
      IPs:
        IP:           fd01:0:0:2::6b
      Controlled By:  ReplicaSet/insights-client-bcd84b656
      Containers:
        client:
          Container ID:   cri-o://157bd7d6bb3a5135762cc8db011684450368b6e60c714fdf20f63ba55a80ac0d
          Image:          e27-h01-000-r650.rdu2.scalelab.redhat.com:5000/acm-d/insights-client-rhel8@sha256:726dbe7df3551344e388eaac66f4c90e14a34ef58002f02ef23fff9ba8cdc344
          Image ID:       e27-h01-000-r650.rdu2.scalelab.redhat.com:5000/acm-d/insights-client-rhel8@sha256:726dbe7df3551344e388eaac66f4c90e14a34ef58002f02ef23fff9ba8cdc344
          Port:           <none>
          Host Port:      <none>
          State:          Waiting
            Reason:       CrashLoopBackOff
          Last State:     Terminated
            Reason:       OOMKilled
            Exit Code:    137
            Started:      Wed, 23 Aug 2023 18:25:15 +0000
            Finished:     Wed, 23 Aug 2023 18:25:19 +0000
          Ready:          False
          Restart Count:  34
          Limits:
            memory:  512Mi
          Requests:
            cpu:      25m
            memory:   32Mi
          Liveness:   exec [ls] delay=20s timeout=10s period=60s #success=1 #failure=3
          Readiness:  exec [ls] delay=15s timeout=10s period=60s #success=1 #failure=3
          Environment:
            USE_MOCK:       false
            CCX_SERVER:     https://console.redhat.com/api/insights-results-aggregator/v1
            POD_NAMESPACE:  open-cluster-management (v1:metadata.namespace)
          Mounts:
            /sslcert from insights-client-certs (rw)
            /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-2lhdj (ro)
      Conditions:
        Type              Status
        Initialized       True
        Ready             False
        ContainersReady   False
        PodScheduled      True
      Volumes:
        insights-client-certs:
          Type:        Secret (a volume populated by a Secret)
          SecretName:  insights-client-certs
          Optional:    false
        kube-api-access-2lhdj:
          Type:                    Projected (a volume that contains injected data from multiple sources)
          TokenExpirationSeconds:  3607
          ConfigMapName:           kube-root-ca.crt
          ConfigMapOptional:       <nil>
          DownwardAPI:             true
          ConfigMapName:           openshift-service-ca.crt
          ConfigMapOptional:       <nil>
      QoS Class:                   Burstable
      Node-Selectors:              <none>
      Tolerations:                 node-role.kubernetes.io/infra:NoSchedule op=Exists
                                   node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                                   node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                                   node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
      Events:
        Type     Reason     Age                  From     Message
        ----     ------     ----                 ----     -------
        Warning  Unhealthy  70m                  kubelet  Readiness probe failed:
        Warning  Unhealthy  70m                  kubelet  Liveness probe failed:
        Normal   Started    35m (x26 over 15h)   kubelet  Started container client
        Normal   Pulled     34m (x26 over 13h)   kubelet  Container image "e27-h01-000-r650.rdu2.scalelab.redhat.com:5000/acm-d/insights-client-rhel8@sha256:726dbe7df3551344e388eaac66f4c90e14a34ef58002f02ef23fff9ba8cdc344" already present on machine
        Normal   Created    34m (x27 over 15h)   kubelet  Created container client
        Warning  BackOff    16s (x293 over 13h)  kubelet  Back-off restarting failed container client in pod insights-client-bcd84b656-jpvng_open-cluster-management(d146df12-63e9-403f-a021-6bd8f8259358)

      Version-Release number of selected component (if applicable):

      OCP Hub and SNO 4.13.9

      ACM - 2.9.0-DOWNSTREAM-2023-08-22-03-59-19

      How reproducible:

      Steps to Reproduce:

      1.  
      2.  
      3. ...

      Actual results:

      Expected results:

      Additional info:

       

      I0823 18:09:39.217910       1 main.go:31] Starting insights-client
      I0823 18:09:39.217955       1 main.go:33] Built from git commit: f0843bcaeb71e8ea58f6427c9518a086b1ae9311
      I0823 18:09:39.218275       1 reportretriever.go:110] Refreshing CRC credentials  
      I0823 18:09:39.218354       1 clustermonitor.go:89] Begin ClusterWatch routine
      I0823 18:09:39.226919       1 clustermonitor.go:136] Starting cluster informer routine for cluster watch for cluster.open-cluster-management.io/v1 resource
      W0823 18:09:39.228227       1 reportretriever.go:97] Unable to get CRC Token: cloud.openshift.com token is not found
      I0823 18:09:43.433991       1 clustermonitor.go:208] Adding vm00926 to all cluster list
      I0823 18:09:43.434010       1 clustermonitor.go:216] Adding vm00926 to Insights cluster list
      I0823 18:09:43.434638       1 clustermonitor.go:208] Adding vm01154 to all cluster list
      I0823 18:09:43.434646       1 clustermonitor.go:216] Adding vm01154 to Insights cluster list
      I0823 18:09:43.435247       1 clustermonitor.go:208] Adding vm01369 to all cluster list
      I0823 18:09:43.435254       1 clustermonitor.go:216] Adding vm01369 to Insights cluster list
      I0823 18:09:43.435839       1 clustermonitor.go:208] Adding vm02527 to all cluster list
      I0823 18:09:43.435846       1 clustermonitor.go:216] Adding vm02527 to Insights cluster list
      I0823 18:09:43.436710       1 clustermonitor.go:208] Adding vm00376 to all cluster list
      I0823 18:09:43.436727       1 clustermonitor.go:216] Adding vm00376 to Insights cluster list
      I0823 18:09:43.437275       1 clustermonitor.go:208] Adding vm00955 to all cluster list
      I0823 18:09:43.437289       1 clustermonitor.go:216] Adding vm00955 to Insights cluster list
      I0823 18:09:43.437819       1 clustermonitor.go:208] Adding vm01010 to all cluster list
      I0823 18:09:43.437826       1 clustermonitor.go:216] Adding vm01010 to Insights cluster list
      I0823 18:09:43.438257       1 clustermonitor.go:208] Adding vm02378 to all cluster list
      I0823 18:09:43.438263       1 clustermonitor.go:216] Adding vm02378 to Insights cluster list
      I0823 18:09:43.438696       1 clustermonitor.go:208] Adding vm03551 to all cluster list
      I0823 18:09:43.438702       1 clustermonitor.go:216] Adding vm03551 to Insights cluster list
      I0823 18:09:43.439468       1 clustermonitor.go:208] Adding vm02986 to all cluster list
      I0823 18:09:43.439477       1 clustermonitor.go:216] Adding vm02986 to Insights cluster list
      I0823 18:09:43.440402       1 clustermonitor.go:208] Adding vm01284 to all cluster list
      I0823 18:09:43.440421       1 clustermonitor.go:216] Adding vm01284 to Insights cluster list
      I0823 18:09:43.441263       1 clustermonitor.go:208] Adding vm01381 to all cluster list
      I0823 18:09:43.441276       1 clustermonitor.go:216] Adding vm01381 to Insights cluster list
      I0823 18:09:43.442121       1 clustermonitor.go:208] Adding vm01592 to all cluster list
      I0823 18:09:43.442133       1 clustermonitor.go:216] Adding vm01592 to Insights cluster list
      I0823 18:09:43.442994       1 clustermonitor.go:208] Adding vm01669 to all cluster list
      I0823 18:09:43.443005       1 clustermonitor.go:216] Adding vm01669 to Insights cluster list
      I0823 18:09:43.443835       1 clustermonitor.go:208] Adding vm01956 to all cluster list
      I0823 18:09:43.443846       1 clustermonitor.go:216] Adding vm01956 to Insights cluster list
      I0823 18:09:43.444675       1 clustermonitor.go:208] Adding vm02084 to all cluster list
      I0823 18:09:43.444686       1 clustermonitor.go:216] Adding vm02084 to Insights cluster list
      I0823 18:09:43.445538       1 clustermonitor.go:208] Adding vm01279 to all cluster list
      I0823 18:09:43.445552       1 clustermonitor.go:216] Adding vm01279 to Insights cluster list
      I0823 18:09:43.446389       1 clustermonitor.go:208] Adding vm01334 to all cluster list
      I0823 18:09:43.446400       1 clustermonitor.go:216] Adding vm01334 to Insights cluster list
      I0823 18:09:43.447237       1 clustermonitor.go:208] Adding vm01811 to all cluster list
      I0823 18:09:43.447248       1 clustermonitor.go:216] Adding vm01811 to Insights cluster list
      I0823 18:09:43.448062       1 clustermonitor.go:208] Adding vm01059 to all cluster list
      I0823 18:09:43.448072       1 clustermonitor.go:216] Adding vm01059 to Insights cluster list
      I0823 18:09:43.448916       1 clustermonitor.go:208] Adding vm01079 to all cluster list
      I0823 18:09:43.448928       1 clustermonitor.go:216] Adding vm01079 to Insights cluster list
      I0823 18:09:43.449766       1 clustermonitor.go:208] Adding vm02232 to all cluster list
      I0823 18:09:43.449777       1 clustermonitor.go:216] Adding vm02232 to Insights cluster list
      I0823 18:09:43.450597       1 clustermonitor.go:208] Adding vm03005 to all cluster list
      I0823 18:09:43.450608       1 clustermonitor.go:216] Adding vm03005 to Insights cluster list
      I0823 18:09:43.451438       1 clustermonitor.go:208] Adding vm03391 to all cluster list
      I0823 18:09:43.451450       1 clustermonitor.go:216] Adding vm03391 to Insights cluster list
      I0823 18:09:43.452293       1 clustermonitor.go:208] Adding vm01867 to all cluster list
      I0823 18:09:43.452304       1 clustermonitor.go:216] Adding vm01867 to Insights cluster list
      I0823 18:09:43.453143       1 clustermonitor.go:208] Adding vm02193 to all cluster list
      I0823 18:09:43.453153       1 clustermonitor.go:216] Adding vm02193 to Insights cluster list
      I0823 18:09:43.453996       1 clustermonitor.go:208] Adding vm02531 to all cluster list
      I0823 18:09:43.454007       1 clustermonitor.go:216] Adding vm02531 to Insights cluster list
      I0823 18:09:43.454878       1 clustermonitor.go:208] Adding vm02557 to all cluster list
      I0823 18:09:43.454889       1 clustermonitor.go:216] Adding vm02557 to Insights cluster list
      I0823 18:09:43.455719       1 clustermonitor.go:208] Adding vm01285 to all cluster list
      I0823 18:09:43.455731       1 clustermonitor.go:216] Adding vm01285 to Insights cluster list
      I0823 18:09:43.456570       1 clustermonitor.go:208] Adding vm02456 to all cluster list
      I0823 18:09:43.456581       1 clustermonitor.go:216] Adding vm02456 to Insights cluster list
      I0823 18:09:43.457405       1 clustermonitor.go:208] Adding vm02768 to all cluster list
      I0823 18:09:43.457415       1 clustermonitor.go:216] Adding vm02768 to Insights cluster list
      I0823 18:09:43.458242       1 clustermonitor.go:208] Adding vm02868 to all cluster list
      I0823 18:09:43.458254       1 clustermonitor.go:216] Adding vm02868 to Insights cluster list
      I0823 18:09:43.459150       1 clustermonitor.go:208] Adding vm00532 to all cluster list
      I0823 18:09:43.459166       1 clustermonitor.go:216] Adding vm00532 to Insights cluster list
      I0823 18:09:43.460067       1 clustermonitor.go:208] Adding vm00661 to all cluster list
      I0823 18:09:43.460079       1 clustermonitor.go:216] Adding vm00661 to Insights cluster list
      I0823 18:09:43.460918       1 clustermonitor.go:208] Adding vm02506 to all cluster list
      I0823 18:09:43.460929       1 clustermonitor.go:216] Adding vm02506 to Insights cluster list
      I0823 18:09:43.466086       1 clustermonitor.go:208] Adding vm03322 to all cluster list
      I0823 18:09:43.466104       1 clustermonitor.go:216] Adding vm03322 to Insights cluster list
      I0823 18:09:43.466971       1 clustermonitor.go:208] Adding vm00942 to all cluster list
      I0823 18:09:43.466983       1 clustermonitor.go:216] Adding vm00942 to Insights cluster list
      I0823 18:09:43.467845       1 clustermonitor.go:208] Adding vm01977 to all cluster list
      I0823 18:09:43.467856       1 clustermonitor.go:216] Adding vm01977 to Insights cluster list
      I0823 18:09:43.468713       1 clustermonitor.go:208] Adding vm03312 to all cluster list
      I0823 18:09:43.468724       1 clustermonitor.go:216] Adding vm03312 to Insights cluster list
      I0823 18:09:43.469589       1 clustermonitor.go:208] Adding vm01596 to all cluster list
      I0823 18:09:43.469602       1 clustermonitor.go:216] Adding vm01596 to Insights cluster list
      I0823 18:09:43.470494       1 clustermonitor.go:208] Adding vm02524 to all cluster list
      I0823 18:09:43.470507       1 clustermonitor.go:216] Adding vm02524 to Insights cluster list
      I0823 18:09:43.471332       1 clustermonitor.go:208] Adding vm01415 to all cluster list
      I0823 18:09:43.471343       1 clustermonitor.go:216] Adding vm01415 to Insights cluster list
      I0823 18:09:43.472207       1 clustermonitor.go:208] Adding vm01610 to all cluster list
      I0823 18:09:43.472218       1 clustermonitor.go:216] Adding vm01610 to Insights cluster list
      I0823 18:09:43.473060       1 clustermonitor.go:208] Adding vm02666 to all cluster list
      I0823 18:09:43.473071       1 clustermonitor.go:216] Adding vm02666 to Insights cluster list
      I0823 18:09:43.473938       1 clustermonitor.go:208] Adding vm03077 to all cluster list
      I0823 18:09:43.473950       1 clustermonitor.go:216] Adding vm03077 to Insights cluster list
      I0823 18:09:43.474798       1 clustermonitor.go:208] Adding vm00391 to all cluster list
      I0823 18:09:43.474810       1 clustermonitor.go:216] Adding vm00391 to Insights cluster list
      I0823 18:09:43.475660       1 clustermonitor.go:208] Adding vm00799 to all cluster list
      I0823 18:09:43.475671       1 clustermonitor.go:216] Adding vm00799 to Insights cluster list
      I0823 18:09:43.476515       1 clustermonitor.go:208] Adding vm01103 to all cluster list
      I0823 18:09:43.476526       1 clustermonitor.go:216] Adding vm01103 to Insights cluster list
      I0823 18:09:43.477367       1 clustermonitor.go:208] Adding vm02576 to all cluster list
      I0823 18:09:43.477378       1 clustermonitor.go:216] Adding vm02576 to Insights cluster list
      I0823 18:09:43.478247       1 clustermonitor.go:208] Adding vm01891 to all cluster list
      I0823 18:09:43.478258       1 clustermonitor.go:216] Adding vm01891 to Insights cluster list
      I0823 18:09:43.479115       1 clustermonitor.go:208] Adding vm01994 to all cluster list
      I0823 18:09:43.479125       1 clustermonitor.go:216] Adding vm01994 to Insights cluster list
      I0823 18:09:43.480015       1 clustermonitor.go:208] Adding vm02145 to all cluster list
      I0823 18:09:43.480027       1 clustermonitor.go:216] Adding vm02145 to Insights cluster list
      I0823 18:09:43.480868       1 clustermonitor.go:208] Adding vm03207 to all cluster list
      I0823 18:09:43.480880       1 clustermonitor.go:216] Adding vm03207 to Insights cluster list
      I0823 18:09:43.481738       1 clustermonitor.go:208] Adding vm03436 to all cluster list
      I0823 18:09:43.481749       1 clustermonitor.go:216] Adding vm03436 to Insights cluster list
      I0823 18:09:43.483150       1 clustermonitor.go:208] Adding vm00831 to all cluster list
      I0823 18:09:43.483164       1 clustermonitor.go:216] Adding vm00831 to Insights cluster list
      I0823 18:09:43.483990       1 clustermonitor.go:208] Adding vm02390 to all cluster list
      I0823 18:09:43.484001       1 clustermonitor.go:216] Adding vm02390 to Insights cluster list
      I0823 18:09:43.484800       1 clustermonitor.go:208] Adding vm00460 to all cluster list
      I0823 18:09:43.484809       1 clustermonitor.go:216] Adding vm00460 to Insights cluster list
      I0823 18:09:43.485616       1 clustermonitor.go:208] Adding vm00838 to all cluster list
      I0823 18:09:43.485627       1 clustermonitor.go:216] Adding vm00838 to Insights cluster list
      I0823 18:09:43.486437       1 clustermonitor.go:208] Adding vm01026 to all cluster list
      I0823 18:09:43.486447       1 clustermonitor.go:216] Adding vm01026 to Insights cluster list
      I0823 18:09:43.487279       1 clustermonitor.go:208] Adding vm01628 to all cluster list
      I0823 18:09:43.487290       1 clustermonitor.go:216] Adding vm01628 to Insights cluster list
      I0823 18:09:43.488114       1 clustermonitor.go:208] Adding vm02598 to all cluster list
      I0823 18:09:43.488124       1 clustermonitor.go:216] Adding vm02598 to Insights cluster list
      I0823 18:09:43.488945       1 clustermonitor.go:208] Adding vm03230 to all cluster list
      I0823 18:09:43.488955       1 clustermonitor.go:216] Adding vm03230 to Insights cluster list
      I0823 18:09:43.489790       1 clustermonitor.go:208] Adding vm02355 to all cluster list
      I0823 18:09:43.489801       1 clustermonitor.go:216] Adding vm02355 to Insights cluster list
      I0823 18:09:43.490605       1 clustermonitor.go:208] Adding vm03317 to all cluster list
      I0823 18:09:43.490615       1 clustermonitor.go:216] Adding vm03317 to Insights cluster list
      I0823 18:09:43.491422       1 clustermonitor.go:208] Adding vm00109 to all cluster list
      I0823 18:09:43.491434       1 clustermonitor.go:216] Adding vm00109 to Insights cluster list
      I0823 18:09:43.492255       1 clustermonitor.go:208] Adding vm00524 to all cluster list
      I0823 18:09:43.492265       1 clustermonitor.go:216] Adding vm00524 to Insights cluster list
      I0823 18:09:43.493073       1 clustermonitor.go:208] Adding vm00853 to all cluster list
      I0823 18:09:43.493084       1 clustermonitor.go:216] Adding vm00853 to Insights cluster list
      I0823 18:09:43.493896       1 clustermonitor.go:208] Adding vm01564 to all cluster list
      I0823 18:09:43.493907       1 clustermonitor.go:216] Adding vm01564 to Insights cluster list
      I0823 18:09:43.494744       1 clustermonitor.go:208] Adding vm03143 to all cluster list
      I0823 18:09:43.494755       1 clustermonitor.go:216] Adding vm03143 to Insights cluster list
      I0823 18:09:43.495574       1 clustermonitor.go:208] Adding vm01005 to all cluster list
      I0823 18:09:43.495585       1 clustermonitor.go:216] Adding vm01005 to Insights cluster list
      I0823 18:09:43.496383       1 clustermonitor.go:208] Adding vm01195 to all cluster list
      I0823 18:09:43.496393       1 clustermonitor.go:216] Adding vm01195 to Insights cluster list
      I0823 18:09:43.497215       1 clustermonitor.go:208] Adding vm01281 to all cluster list
      I0823 18:09:43.497226       1 clustermonitor.go:216] Adding vm01281 to Insights cluster list
      I0823 18:09:43.498035       1 clustermonitor.go:208] Adding vm00689 to all cluster list
      I0823 18:09:43.498045       1 clustermonitor.go:216] Adding vm00689 to Insights cluster list
      I0823 18:09:43.498846       1 clustermonitor.go:208] Adding vm01324 to all cluster list
      I0823 18:09:43.498857       1 clustermonitor.go:216] Adding vm01324 to Insights cluster list
      I0823 18:09:43.499679       1 clustermonitor.go:208] Adding vm01727 to all cluster list
      I0823 18:09:43.499690       1 clustermonitor.go:216] Adding vm01727 to Insights cluster list
      I0823 18:09:43.500599       1 clustermonitor.go:208] Adding vm02493 to all cluster list
      I0823 18:09:43.500611       1 clustermonitor.go:216] Adding vm02493 to Insights cluster list
      I0823 18:09:43.501423       1 clustermonitor.go:208] Adding vm02881 to all cluster list
      I0823 18:09:43.501433       1 clustermonitor.go:216] Adding vm02881 to Insights cluster list
      I0823 18:09:43.502278       1 clustermonitor.go:208] Adding vm03316 to all cluster list
      I0823 18:09:43.502289       1 clustermonitor.go:216] Adding vm03316 to Insights cluster list
      I0823 18:09:43.503144       1 clustermonitor.go:208] Adding vm01328 to all cluster list
      I0823 18:09:43.503156       1 clustermonitor.go:216] Adding vm01328 to Insights cluster list
      I0823 18:09:43.504003       1 clustermonitor.go:208] Adding vm03585 to all cluster list
      I0823 18:09:43.504014       1 clustermonitor.go:216] Adding vm03585 to Insights cluster list
      I0823 18:09:43.504857       1 clustermonitor.go:208] Adding vm00897 to all cluster list
      I0823 18:09:43.504868       1 clustermonitor.go:216] Adding vm00897 to Insights cluster list
      I0823 18:09:43.505743       1 clustermonitor.go:208] Adding vm01653 to all cluster list
      I0823 18:09:43.505754       1 clustermonitor.go:216] Adding vm01653 to Insights cluster list
      I0823 18:09:43.506607       1 clustermonitor.go:208] Adding vm01923 to all cluster list
      I0823 18:09:43.506617       1 clustermonitor.go:216] Adding vm01923 to Insights cluster list
      I0823 18:09:43.507469       1 clustermonitor.go:208] Adding vm02268 to all cluster list
      I0823 18:09:43.507481       1 clustermonitor.go:216] Adding vm02268 to Insights cluster list
      I0823 18:09:43.508313       1 clustermonitor.go:208] Adding vm02371 to all cluster list
      I0823 18:09:43.508323       1 clustermonitor.go:216] Adding vm02371 to Insights cluster list
      I0823 18:09:43.509169       1 clustermonitor.go:208] Adding vm03406 to all cluster list
      I0823 18:09:43.509180       1 clustermonitor.go:216] Adding vm03406 to Insights cluster list
      I0823 18:09:43.510018       1 clustermonitor.go:208] Adding vm00705 to all cluster list
      I0823 18:09:43.510029       1 clustermonitor.go:216] Adding vm00705 to Insights cluster list
      I0823 18:09:43.510895       1 clustermonitor.go:208] Adding vm00245 to all cluster list
      I0823 18:09:43.510905       1 clustermonitor.go:216] Adding vm00245 to Insights cluster list
      I0823 18:09:43.511747       1 clustermonitor.go:208] Adding vm02143 to all cluster list
      I0823 18:09:43.511758       1 clustermonitor.go:216] Adding vm02143 to Insights cluster list
      I0823 18:09:43.512611       1 clustermonitor.go:208] Adding vm02664 to all cluster list
      I0823 18:09:43.512620       1 clustermonitor.go:216] Adding vm02664 to Insights cluster list
      I0823 18:09:43.513475       1 clustermonitor.go:208] Adding vm03403 to all cluster list
      I0823 18:09:43.513485       1 clustermonitor.go:216] Adding vm03403 to Insights cluster list
      I0823 18:09:43.514331       1 clustermonitor.go:208] Adding vm00170 to all cluster list
      I0823 18:09:43.514342       1 clustermonitor.go:216] Adding vm00170 to Insights cluster list
      I0823 18:09:43.515190       1 clustermonitor.go:208] Adding vm01782 to all cluster list
      I0823 18:09:43.515201       1 clustermonitor.go:216] Adding vm01782 to Insights cluster list
      I0823 18:09:43.517027       1 clustermonitor.go:208] Adding vm02759 to all cluster list
      I0823 18:09:43.517041       1 clustermonitor.go:216] Adding vm02759 to Insights cluster list
      I0823 18:09:43.517911       1 clustermonitor.go:208] Adding vm03158 to all cluster list
      I0823 18:09:43.517922       1 clustermonitor.go:216] Adding vm03158 to Insights cluster list
      I0823 18:09:43.518767       1 clustermonitor.go:208] Adding vm03372 to all cluster list
      I0823 18:09:43.518779       1 clustermonitor.go:216] Adding vm03372 to Insights cluster list
      I0823 18:09:43.519636       1 clustermonitor.go:208] Adding vm03496 to all cluster list
      I0823 18:09:43.519647       1 clustermonitor.go:216] Adding vm03496 to Insights cluster list
      I0823 18:09:43.520531       1 clustermonitor.go:208] Adding vm00747 to all cluster list
      I0823 18:09:43.520541       1 clustermonitor.go:216] Adding vm00747 to Insights cluster list
      I0823 18:09:43.521391       1 clustermonitor.go:208] Adding vm01704 to all cluster list
      I0823 18:09:43.521405       1 clustermonitor.go:216] Adding vm01704 to Insights cluster list
      I0823 18:09:43.522616       1 clustermonitor.go:208] Adding vm03301 to all cluster list
      I0823 18:09:43.522630       1 clustermonitor.go:216] Adding vm03301 to Insights cluster list
      I0823 18:09:43.523503       1 clustermonitor.go:208] Adding vm00096 to all cluster list
      I0823 18:09:43.523515       1 clustermonitor.go:216] Adding vm00096 to Insights cluster list
      I0823 18:09:43.524385       1 clustermonitor.go:208] Adding vm00171 to all cluster list
      I0823 18:09:43.524396       1 clustermonitor.go:216] Adding vm00171 to Insights cluster list

            xdharmai@redhat.com XAVIER RAJESH DHARMAIYAN
            akrzos@redhat.com Alex Krzos
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: