[MIG-1661] Performing StorageClass conversion triggers the scale-down of all aplications of the namespace

Type: Bug
Resolution: Unresolved
Priority: Normal
Fix Version/s: MTC 1.8.7
Affects Version/s: None
Component/s: None
Labels:
None

Blocked:
False
Blocked Reason:

Hide

None

Show
None
Ready:
False
QEStatus:
ToDo
Intelligence Requested:
Market:

Customer Impact:

Customer Escalated

SFDC Cases Links:
SFDC Cases Open:
SFDC Cases Counter:

Environment:

While running a StorageClass conversion was noticed that if there's more than one application running, then MTC scales down all the applications on the cutover phase, even those noninvolved in the migration.

- - Sort By Name
  - Sort By Date
  - Ascending
  - Descending
  - Thumbnails
  - List
  - Download All

image-2024-10-17-15-57-08-858.png
75 kB
2024/10/17 1:57 PM
image-2024-10-17-15-58-36-411.png
25 kB
2024/10/17 1:58 PM

links to

openshift/openshift-docs#84677: MIG-1664: MTC-1.8.5-release-notes

Luis Perez Besa added a comment - 2024/10/17 1:59 PM

Reproducer:
a) Two statefulset applications running on the same namespace:

$ oc get statefulset --show-labels -n papagayo
NAME        READY   AGE   LABELS
test-mig1   1/1     24s   app=test-mig1
test-mig2   1/1     21s   app=test-mig2

$ oc get pod -o wide --show-labels -n papagayo
NAME          READY   STATUS    RESTARTS   AGE   IP             NODE                                     NOMINATED NODE   READINESS GATES   LABELS
test-mig1-0   1/1     Running   0          48s   10.130.3.149   lperezbe-cluster1-kc5g4-worker-0-qgpfw   <none>           <none>            app=test-mig1,apps.kubernetes.io/pod-index=0,controller-revision-hash=test-mig1-6758bd6fb7,statefulset.kubernetes.io/pod-name=test-mig1-0
test-mig2-0   1/1     Running   0          45s   10.131.2.218   lperezbe-cluster1-kc5g4-worker-0-nwnkv   <none>           <none>            app=test-mig2,apps.kubernetes.io/pod-index=0,controller-revision-hash=test-mig2-78fd469d54,statefulset.kubernetes.io/pod-name=test-mig2-0

b) Running Storageclass conversion just for the PVC of a single application:

c) Executing Cutover triggers that both go down:

Even if the test-mig2 app was not involved in this migration

$ oc get pod -w -n papagayo
NAME          READY   STATUS    RESTARTS   AGE
test-mig1-0   1/1     Running   0          3m52s
test-mig2-0   1/1     Running   0          3m49s
test-mig1-0   1/1     Terminating   0          4m9s
test-mig2-0   1/1     Terminating   0          4m6s
test-mig1-0   0/1     Terminating   0          4m39s
test-mig2-0   0/1     Terminating   0          4m36s
test-mig2-0   0/1     Terminating   0          4m36s
test-mig2-0   0/1     Terminating   0          4m36s
test-mig2-0   0/1     Terminating   0          4m36s
test-mig1-0   0/1     Terminating   0          4m39s
test-mig1-0   0/1     Terminating   0          4m39s
test-mig1-0   0/1     Terminating   0          4m39s
rsync-server   0/2     Pending       0          0s
rsync-server   0/2     Pending       0          1s
rsync-server   0/2     Pending       0          1s
rsync-server   0/2     ContainerCreating   0          1s
rsync-server   0/2     ContainerCreating   0          5s
rsync-server   2/2     Running             0          5s
rsync-s7j8l    0/2     Pending             0          0s
rsync-s7j8l    0/2     Pending             0          0s
rsync-s7j8l    0/2     Pending             0          0s
rsync-s7j8l    0/2     ContainerCreating   0          0s
rsync-s7j8l    0/2     ContainerCreating   0          11s
rsync-s7j8l    1/2     NotReady            0          12s
rsync-s7j8l    0/2     Completed           0          13s
rsync-s7j8l    0/2     Completed           0          14s
rsync-s7j8l    0/2     Completed           0          15s
rsync-s7j8l    0/2     Completed           0          19s
rsync-s7j8l    0/2     Terminating         0          19s
rsync-s7j8l    0/2     Terminating         0          19s
rsync-server   2/2     Terminating         0          25s
rsync-server   0/2     Terminating         0          26s
rsync-server   0/2     Terminating         0          27s
rsync-server   0/2     Terminating         0          27s
rsync-server   0/2     Terminating         0          27s
test-mig1-0    0/1     Pending             0          0s
test-mig1-0    0/1     Pending             0          0s
test-mig1-0    0/1     Pending             0          0s
test-mig1-0    0/1     ContainerCreating   0          0s
test-mig2-0    0/1     Pending             0          0s
test-mig2-0    0/1     Pending             0          1s
test-mig2-0    0/1     Pending             0          1s
test-mig2-0    0/1     ContainerCreating   0          1s
test-mig1-0    0/1     ContainerCreating   0          6s
test-mig1-0    1/1     Running             0          8s
test-mig2-0    0/1     ContainerCreating   0          9s
test-mig2-0    1/1     Running             0          10s

Luis Perez Besa added a comment - 2024/10/17 1:59 PM Reproducer: a) Two statefulset applications running on the same namespace: $ oc get statefulset --show-labels -n papagayo NAME READY AGE LABELS test-mig1 1/1 24s app=test-mig1 test-mig2 1/1 21s app=test-mig2 $ oc get pod -o wide --show-labels -n papagayo NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS test-mig1-0 1/1 Running 0 48s 10.130.3.149 lperezbe-cluster1-kc5g4-worker-0-qgpfw <none> <none> app=test-mig1,apps.kubernetes.io/pod-index=0,controller-revision-hash=test-mig1-6758bd6fb7,statefulset.kubernetes.io/pod-name=test-mig1-0 test-mig2-0 1/1 Running 0 45s 10.131.2.218 lperezbe-cluster1-kc5g4-worker-0-nwnkv <none> <none> app=test-mig2,apps.kubernetes.io/pod-index=0,controller-revision-hash=test-mig2-78fd469d54,statefulset.kubernetes.io/pod-name=test-mig2-0 b) Running Storageclass conversion just for the PVC of a single application: c) Executing Cutover triggers that both go down: Even if the test-mig2 app was not involved in this migration $ oc get pod -w -n papagayo NAME READY STATUS RESTARTS AGE test-mig1-0 1/1 Running 0 3m52s test-mig2-0 1/1 Running 0 3m49s test-mig1-0 1/1 Terminating 0 4m9s test-mig2-0 1/1 Terminating 0 4m6s test-mig1-0 0/1 Terminating 0 4m39s test-mig2-0 0/1 Terminating 0 4m36s test-mig2-0 0/1 Terminating 0 4m36s test-mig2-0 0/1 Terminating 0 4m36s test-mig2-0 0/1 Terminating 0 4m36s test-mig1-0 0/1 Terminating 0 4m39s test-mig1-0 0/1 Terminating 0 4m39s test-mig1-0 0/1 Terminating 0 4m39s rsync-server 0/2 Pending 0 0s rsync-server 0/2 Pending 0 1s rsync-server 0/2 Pending 0 1s rsync-server 0/2 ContainerCreating 0 1s rsync-server 0/2 ContainerCreating 0 5s rsync-server 2/2 Running 0 5s rsync-s7j8l 0/2 Pending 0 0s rsync-s7j8l 0/2 Pending 0 0s rsync-s7j8l 0/2 Pending 0 0s rsync-s7j8l 0/2 ContainerCreating 0 0s rsync-s7j8l 0/2 ContainerCreating 0 11s rsync-s7j8l 1/2 NotReady 0 12s rsync-s7j8l 0/2 Completed 0 13s rsync-s7j8l 0/2 Completed 0 14s rsync-s7j8l 0/2 Completed 0 15s rsync-s7j8l 0/2 Completed 0 19s rsync-s7j8l 0/2 Terminating 0 19s rsync-s7j8l 0/2 Terminating 0 19s rsync-server 2/2 Terminating 0 25s rsync-server 0/2 Terminating 0 26s rsync-server 0/2 Terminating 0 27s rsync-server 0/2 Terminating 0 27s rsync-server 0/2 Terminating 0 27s test-mig1-0 0/1 Pending 0 0s test-mig1-0 0/1 Pending 0 0s test-mig1-0 0/1 Pending 0 0s test-mig1-0 0/1 ContainerCreating 0 0s test-mig2-0 0/1 Pending 0 0s test-mig2-0 0/1 Pending 0 1s test-mig2-0 0/1 Pending 0 1s test-mig2-0 0/1 ContainerCreating 0 1s test-mig1-0 0/1 ContainerCreating 0 6s test-mig1-0 1/1 Running 0 8s test-mig2-0 0/1 ContainerCreating 0 9s test-mig2-0 1/1 Running 0 10s

Details

Description

Attachments

Attachments

Issue Links

Easy Agile Planning Poker

Activity

Collapse comment: Luis Perez Besa added a comment - 2024/10/17 1:59 PM

Expand comment: Luis Perez Besa added a comment - 2024/10/17 1:59 PM

People

Dates