Uploaded image for project: 'OpenShift Etcd'
  1. OpenShift Etcd
  2. ETCD-674

Add e2e test for scaling when a member is unhealthy

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • None
    • None
    • 5
    • False
    • None
    • False
    • ETCD Sprint 259, ETCD Sprint 260, ETCD Sprint 261, ETCD Sprint 262

      Add two E2E test for vertical scaling scenario: when a member is unhealthy and another scenario when kubelet is not working on a node.

      First test validates that scale down happens before scale up if the deleted member is unhealthy.CPMS is disabled to observe that scale-down happens first in this case.

      1. If the CPMS is active, first disable it by deleting the CPMS custom resource.
      2. Remove the static pod manifest from a node and stop the kubelet on the node. This makes the member unhealthy.
      3. Delete the machine hosting the node in step 2.
      4. Verify the member removal and the total voting member count of 2 to ensure scale-down happens first when a member is unhealthy.
      5. Restore the initial cluster state by creating a new machine(scale-up) and re-enabling CPMS.

      The second test covers a vertical scaling scenario when kubelet is not working on a node.
      This test validates that deleting the machine hosting the node where the kubelet is stopped doesn't get stuck when CPMS is enabled. The case in this bug: https://issues.redhat.com/browse/OCPBUGS-17199
      CPMS should be active for this test scenario

      1. Stop the kubelet on a node.
      2. Delete the machine hosting the node in step 2.
      3. That should prompt the ControlPlaneMachineSetOperator(CPMSO) to create a replacement machine and node for that machine index.
      4. The operator will first scale-up the new machine's member.
      5. Then scale-down the machine that is pending deletion by removing its member and deletion hook.

              rh-ee-jujohn Jubitta John
              rhn-coreos-htariq Haseeb Tariq
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: