CRYPT_ACTIVATE_SERIALIZE_MEMORY_HARD_PBKDF activation flag is ignored with cryptsetup 2.8.1. This is regression in newer builds with upstream cryptsetup-2.8.x.

This is a regression in introduced in (future) RHEL 10.2. But also the flag was never respected when the device was activated for example by LUKS2 tokens.

---------------------------------------------

To test the issue:

1) create LUKS2 device using Argon2 kdf with high enough memory requirements (e.g. at least 1GiB): "cryptsetup luksFormat /dev/device --pbkdf Argon2id --pbkdf-memory 1048576" (the --pbkdf-memory argument takes KiB as the base unit).

2) run multiple instances of the following command in a cgroup with a limited memory. For lets have a cgroup with a 3 GiB hard memory limit and run 5 instances of the following command in-parallel: "cryptsetup open /dev/device --test-passphrase --key-file /file/with/valid/passphrase &"

On the builds containing the regression it should fail with "Not enough memory" error since there will not be enough memory available in the cgroup to run all commands in parallel.

On the fixed build, it should pass since all the commands will run serialized on the memory hard lock.

To test the bug with LUKS2 tokens:

1) upload the LUKS2 passphrase in a keyring by "keyctl add user myTestKeyDescription <VALID_PASSPHRASE> @s"

2) add LUKS2 keyring token by: "cryptsetup token add --key-description myTestKeyDescription /dev/device"

3) run multiple instances of "cryptsetup open --token-only --test-passphrase /dev/device" in cgroup.