-
Bug
-
Resolution: Done-Errata
-
Normal
-
rhel-9.0.0
-
pcs-0.11.6-6.el9
-
None
-
Moderate
-
25e28913ae7ac07cdfa8c39c3c4920b4823839e7 25a5a4bb4b1499b425707c5e736bd183ee97ee8d 3a456616f5e9453e59d7b6f712444692d6a4ad07 c2c7c033aaff8752587959946441fde84e10dff9 a74e35a30c35e00cbcc680cac4241ff8922db086 10e637e96f01ed78667a829344ddacd098b47a6d
-
sst_high_availability
-
ssg_filesystems_storage_and_HA
-
13
-
19
-
5
-
QE ack
-
False
-
-
Yes
-
None
-
Pass
-
None
-
Bug Fix
-
-
Done
-
-
Unspecified
-
None
+++ This bug was initially created as a clone of Bug #1578789 +++
Description of problem:
- moving and banning an inner bundle resource
- pcs should exit with an error saying moving / banning inner bundle resources is not allowed
- moving / banning a pacemaker internal bundle resource
- works OK, pcs says the resource doesn't exist (it's not in the CIB)
- moving and banning a bundle resource
- does not currently work in crm_resource even for 1-instance bundles:
Resource 'DummyBundle' not moved: active in 2 locations.
- does not currently work in crm_resource even for 1-instance bundles:
- moving and banning a clone resource
- works in crm_resource for 1-instance clones for both a clone and its inner resource
- for multi-instance clones, moving a resource in a clone creates a constraint, whereas moving a clone resource exits with an error
- pcs should not allow moving / banning a resource in a clone, then it can simply run crm_resource every time and based on its output inform a user if a clone could or couldn't be moved / banned
- another option is to automatically switch to moving / banning a bundle / clone if the user tries to move / ban a resource in the bundle / clone
Version-Release number of selected component (if applicable):
pcs-0.9.162-5.el7.x86_64
How reproducible:
always, easily
Steps to Reproduce:
Performing tests of move without node specification
dummy0 is a resource inside DummyBundle bundle with replicas=1
[root@virt-143 ~]# pcs resource move dummy0 Warning: Creating location constraint cli-ban-dummy0-on-DummyBundle-0 with a score of -INFINITY for resource dummy0 on node DummyBundle-0. This will prevent dummy0 from running on DummyBundle-0 until the constraint is removed. This will be the case even if DummyBundle-0 is the last node in the cluster. [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy0 Disabled on: DummyBundle-0 (score:-INFINITY) (role: Started) (id:cli-ban-dummy0-on-DummyBundle-0) [root@virt-143 ~]# crm_resource --resource dummy0 --move WARNING: Creating rsc_location constraint 'cli-ban-dummy0-on-DummyBundle-0' with a score of -INFINITY for resource dummy0 on DummyBundle-0. This will prevent dummy0 from running on DummyBundle-0 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if DummyBundle-0 is the last node in the cluster This message can be disabled with --quiet [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy0 Disabled on: DummyBundle-0 (score:-INFINITY) (role: Started) (id:cli-ban-dummy0-on-DummyBundle-0)
Constraint has been created, but resource WAS NOT moved. The reason is obvious as DummyBundle-0 is not an actual node. This is a possible place for improvement.
[root@virt-143 ~]# pcs resource move DummyBundle-0 Error: DummyBundle-0 is not a valid resource [root@virt-143 ~]# crm_resource --resource DummyBundle-0 --move WARNING: Creating rsc_location constraint 'cli-ban-DummyBundle-0-on-virt-143' with a score of -INFINITY for resource DummyBundle-0 on virt-143. This will prevent DummyBundle-0 from running on virt-143 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if virt-143 is the last node in the cluster This message can be disabled with --quiet Error performing operation: Update does not conform to the configured schema
Neither pcs not crm_resource is able to target the DummyBundle-0 pseudo resource which is expected.
[root@virt-143 ~]# pcs resource move DummyBundle Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource DummyBundle --move Resource 'DummyBundle' not moved: active in 2 locations. You can prevent 'DummyBundle' from running on a specific location with: --ban --node <name> Error performing operation: Invalid argument
In this situation pcs simply shouts for trying to move a clone, while crm_resource gives us something else - a misinformation about the number of locations being active. There should only be one active due to replicas=1. Let's see how well the pure clones will do:
dummy1 is a clone with clone-max=1
[root@virt-143 ~]# pcs resource move dummy1 Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource dummy1 --move WARNING: Creating rsc_location constraint 'cli-ban-dummy1-on-virt-145' with a score of -INFINITY for resource dummy1 on virt-145. This will prevent dummy1 from running on virt-145 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if virt-145 is the last node in the cluster This message can be disabled with --quiet [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Disabled on: virt-145 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-on-virt-145)
A clone with single instance was refused to move by pcs, but actually passed and WAS moved with crm_resource. There might be a room for improvement in pcs.
[root@virt-143 ~]# pcs resource move dummy1-clone Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource dummy1-clone --move WARNING: Creating rsc_location constraint 'cli-ban-dummy1-clone-on-virt-143' with a score of -INFINITY for resource dummy1-clone on virt-143. This will prevent dummy1-clone from running on virt-143 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if virt-143 is the last node in the cluster This message can be disabled with --quiet [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1-clone Disabled on: virt-143 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-clone-on-virt-143)
When targeting clone ID, once again a clone with single instance was refused to move by pcs, but passed and WAS moved with crm_resource.
dummy1 is a clone with clone-max=2
[root@virt-143 ~]# pcs resource move dummy1 Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource dummy1 --move WARNING: Creating rsc_location constraint 'cli-ban-dummy1-on-virt-143' with a score of -INFINITY for resource dummy1 on virt-143. This will prevent dummy1 from running on virt-143 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if virt-143 is the last node in the cluster This message can be disabled with --quiet [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Disabled on: virt-143 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-on-virt-143) [root@virt-143 ~]# crm_resource --resource dummy1 --move WARNING: Creating rsc_location constraint 'cli-ban-dummy1-on-virt-145' with a score of -INFINITY for resource dummy1 on virt-145. This will prevent dummy1 from running on virt-145 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if virt-145 is the last node in the cluster This message can be disabled with --quiet [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Disabled on: virt-143 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-on-virt-143) Disabled on: virt-145 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-on-virt-145)
With more than one clone instance crm_resource does surprisingly perform a constraint-based move by moving ONE of the instance. Even if ran twice in a row.
[root@virt-143 ~]# pcs resource move dummy1-clone Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource dummy1-clone --move Resource 'dummy1-clone' not moved: active in 2 locations. You can prevent 'dummy1-clone' from running on a specific location with: --ban --node <name> Error performing operation: Invalid argument
Here with clone ID and two instances crm_resource won't do us a favor anymore. Pcs keeps printing the same error as before.
So, the conclusion for 'resource move':
- pcs is not really up in sync with crm_resource behavior in terms of being able to move a single clone instance
- support for moving a bundle in crm_resource is not on par with being able to move a clone, at least in case of just one instance being present due to:
- 2 active locations being improperly reported
- the move constraint of a primitive is incorrectly targeted to a non-existent DummyBundle-0 node.
It is worth pointing out that creating a pure location constraint for DummyBundle will move the only existing bundle instance to the desired node as one would expect. I yet have to test the behavior of 'resource ban'.
Performing tests of move with node specification
dummy0 is a resource inside DummyBundle bundle with replicas=1
[root@virt-143 ~]# pcs resource move dummy0 virt-145 [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy0 Enabled on: virt-145 (score:INFINITY) (role: Started) (id:cli-prefer-dummy0) [root@virt-143 ~]# crm_resource --resource dummy0 --move --node virt-145 [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy0 Enabled on: virt-145 (score:INFINITY) (role: Started) (id:cli-prefer-dummy0)
Constraint has been created, but resource WAS NOT moved.
[root@virt-143 ~]# pcs resource move DummyBundle-0 virt-145 Error: DummyBundle-0 is not a valid resource [root@virt-143 ~]# crm_resource --resource DummyBundle-0 --move --node virt-145 Error performing operation: Update does not conform to the configured schema
Neither pcs not crm_resource is able to target the DummyBundle-0 pseudo resource which is expected.
[root@virt-143 ~]# pcs resource move DummyBundle virt-145 Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource DummyBundle --move --node virt-145 Resource 'DummyBundle' not moved: active on multiple nodes Error performing operation: Invalid argument
Just like in the previous case the crm_resource gives us a misinformation about the number of locations being active. There should only be one active due to replicas=1.
dummy1 is a clone with clone-max=1
[root@virt-143 ~]# pcs resource move dummy1 virt-145 Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource dummy1 --move --node virt-145 [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Enabled on: virt-145 (score:INFINITY) (role: Started) (id:cli-prefer-dummy1)
A clone with single instance was refused to move by pcs, but actually passed and WAS moved with crm_resource. There might be a room for improvement in pcs.
[root@virt-143 ~]# pcs resource move dummy1-clone virt-146 Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource dummy1-clone --move --node virt-146 [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1-clone Enabled on: virt-146 (score:INFINITY) (role: Started) (id:cli-prefer-dummy1-clone)
When targeting clone ID, once again a clone with single instance was refused to move by pcs, but passed and WAS moved with crm_resource.
dummy1 is a clone with clone-max=2
[root@virt-143 ~]# pcs resource move dummy1 virt-145 Error: cannot move cloned resources [root@virt-143 ~]# crm_resource --resource dummy1 --move --node virt-145 [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Enabled on: virt-145 (score:INFINITY) (role: Started) (id:cli-prefer-dummy1)
With more than one clone instance crm_resource does perform a constraint-based move by moving ONE of the instance.
[root@virt-143 ~]# pcs resource move dummy1-clone virt-146 Error: cannot move cloned resources [root@virt-143 ~]# [root@virt-143 ~]# crm_resource --resource dummy1-clone --move --node virt-146 [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1-clone Enabled on: virt-146 (score:INFINITY) (role: Started) (id:cli-prefer-dummy1-clone)
Here with clone ID and two instances crm_resource won't do us a favor anymore. Pcs keeps printing the same error as before.
The conclusion for 'resource move' with node specification is pretty much aligned with the previous one:
- pcs is not really up in sync with crm_resource behavior in terms of being able to move a single clone instance
- support for moving a bundle in crm_resource is not on par with being able to move a clone, at least in case of just one instance being present due to:
- 2 active locations being improperly reported
- the move constraint of a primitive is incorrectly targeted to a non-existent DummyBundle-0 node.
Performing tests of ban with crm_resource is pretty much identical to simple move (with no node specification), so I am going to mention pcs differences only:
dummy0 is a resource inside DummyBundle bundle with replicas=1
[root@virt-143 ~]# pcs resource ban DummyBundle Error: error moving/banning/clearing resource Resource 'DummyBundle' not moved: active in 2 locations. You can prevent 'DummyBundle' from running on a specific location with: --ban --node <name> Error performing operation: Invalid argument [root@virt-143 ~]# crm_resource --resource DummyBundle --ban Resource 'DummyBundle' not moved: active in 2 locations. You can prevent 'DummyBundle' from running on a specific location with: --ban --node <name> Error performing operation: Invalid argument
The difference is in pcs output when targetting DummyBundle which no longer complains about a clone but instead passes the crm_resource message as is. We'll need a more user friendly message in pcs.
dummy1 is a clone with clone-max=1
[root@virt-143 ~]# pcs resource ban dummy1 Warning: Creating location constraint cli-ban-dummy1-on-virt-143 with a score of -INFINITY for resource dummy1 on node virt-143. This will prevent dummy1 from running on virt-143 until the constraint is removed. This will be the case even if virt-143 is the last node in the cluster. [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Disabled on: virt-143 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-on-virt-143) [root@virt-143 ~]# crm_resource --resource dummy1 --ban WARNING: Creating rsc_location constraint 'cli-ban-dummy1-on-virt-144' with a score of -INFINITY for resource dummy1 on virt-144. This will prevent dummy1 from running on virt-144 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if virt-144 is the last node in the cluster This message can be disabled with --quiet [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Disabled on: virt-144 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-on-virt-144)
The resource WAS moved in both cases so pcs and crm_resource are in sync regarding behavior.
[root@virt-143 ~]# pcs resource ban dummy1-clone Warning: Creating location constraint cli-ban-dummy1-clone-on-virt-143 with a score of -INFINITY for resource dummy1-clone on node virt-143. This will prevent dummy1-clone from running on virt-143 until the constraint is removed. This will be the case even if virt-143 is the last node in the cluster. [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1-clone Disabled on: virt-143 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-clone-on-virt-143) [root@virt-143 ~]# crm_resource --resource dummy1-clone --ban WARNING: Creating rsc_location constraint 'cli-ban-dummy1-clone-on-virt-144' with a score of -INFINITY for resource dummy1-clone on virt-144. This will prevent dummy1-clone from running on virt-144 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if virt-144 is the last node in the cluster This message can be disabled with --quiet [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1-clone Disabled on: virt-144 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-clone-on-virt-144)
Again the resource WAS moved in both cases so pcs and crm_resource are in sync regarding behavior.
dummy1 is a clone with clone-max=2
[root@virt-143 ~]# pcs resource ban dummy1 Warning: Creating location constraint cli-ban-dummy1-on-virt-145 with a score of -INFINITY for resource dummy1 on node virt-145. This will prevent dummy1 from running on virt-145 until the constraint is removed. This will be the case even if virt-145 is the last node in the cluster. [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Disabled on: virt-145 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-on-virt-145) [root@virt-143 ~]# crm_resource --resource dummy1 --ban WARNING: Creating rsc_location constraint 'cli-ban-dummy1-on-virt-143' with a score of -INFINITY for resource dummy1 on virt-143. This will prevent dummy1 from running on virt-143 until the constraint is removed using the 'crm_resource --clear' command or manually with cibadmin This will be the case even if virt-143 is the last node in the cluster This message can be disabled with --quiet [root@virt-143 ~]# pcs constraint --full Location Constraints: Resource: dummy1 Disabled on: virt-143 (score:-INFINITY) (role: Started) (id:cli-ban-dummy1-on-virt-143)
One of the resource instances WAS moved in both cases so pcs and crm_resource are in sync regarding behavior.
[root@virt-143 ~]# pcs resource ban dummy1-clone Error: error moving/banning/clearing resource Resource 'dummy1-clone' not moved: active in 2 locations. You can prevent 'dummy1-clone' from running on a specific location with: --ban --node <name> Error performing operation: Invalid argument [root@virt-143 ~]# [root@virt-143 ~]# crm_resource --resource dummy1-clone --ban Resource 'dummy1-clone' not moved: active in 2 locations. You can prevent 'dummy1-clone' from running on a specific location with: --ban --node <name> Error performing operation: Invalid argument
The difference is in pcs output when targetting DummyBundle which no longer complains about a clone but instead passes the crm_resource message as is. We'll need a more user friendly message in pcs.
The conclusion for 'resource ban' is the same for crm_resource, things are only different for pcs:
- pcs now doesn't scream about clones anymore but instead pass crm_resource's error message directly, which we might need to modify a little
- support for banning a bundle in crm_resource is not on par with being able to move a clone, at least in case of just one instance being present due to:
- 2 active locations being improperly reported
- the move constraint of a primitive is incorrectly targeted to a non-existent DummyBundle-0 node
— Additional comment from Tomas Jelinek on 2023-07-25 15:23:54 CEST —
Summary after bz1578820 has been resolved:
'pcs resource ban' works as described in comment 0. Such behavior is fine and there is nothing to be fixed. Pcs delegates the banning to crm_resource, including detection of number of location a resource is active in.
'pcs resource move-with constraint' and 'pcs resource move' needs to be fixed:
- Moving / banning an inner bundle resource from its bundle should be disallowed in pcs. Currently, pcs doesn't check for this and creates a constraint. The constraint has no effect, as expected.
- Moving a 1-replica bundle works, but pcs does not allow it. There is a check in a validator in 'pcs resource move' command which prevents moving all bundle resources. As moving a 1-replica bundle works in pacemaker, and moving multi-replica bundles is prevented by pacemaker, the check in pcs can be removed completely.
- Moving a clone resource works the same as moving a bundle resource and is disallowed by pcs in the same way as well. Removing this limitation from pcs should be considered.
- blocks
-
RHEL-7584 Fix moving / banning clone and bundle resources
- Closed
- external trackers
- links to
-
RHBA-2023:124598 pcs bug fix and enhancement update
- mentioned on