Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-5263

In FIPS mode, NSS DH_NewKey() fails because RNG_RNGInit() was never called

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • rhel-8.5.0
    • nss
    • Major
    • sst_security_crypto
    • ssg_security
    • False
    • Hide

      None

      Show
      None
    • If docs needed, set a value

      Description of problem:
      On ppc64le in FIPS mode, DH_NewKey() fails when used by ACVP tests. This happens because the random number generator is not initialized, which is usually performed by sftk_startup_tests in lib/softoken/fipstest.c, which calls RNG_RNGInit().

      On x86_64, sftk_startup_tests is automatically run when the library is loaded, because its declaration is marked with _attribute_((constructor)) (see https://gcc.gnu.org/onlinedocs/gcc-12.2.0/gcc/Common-Function-Attributes.html#index-constructor-function-attribute).

      The code only does this if NSS_NO_INIT_SUPPORT is not defined. However, lib/softoken/Makefile contains this code:

      ifeq ($(OS_TARGET),Linux)
      ifeq ($(CPU_ARCH),ppc)
      ifdef USE_64
      DEFINES += -DNSS_NO_INIT_SUPPORT
      endif # USE_64
      endif # ppc
      endif # Linux

      git blame suggests that this was added by Bob 6 years ago, and is still present in upstream tip. I am not sure why, the commit message doesn't say:

      Bug 1181814 - Pick up FIPS-140 certification work done by Red Hat, r=kaie, emaldona
      Includes modifications submitted by kaie and emaldona, r=rrelyea

      I did verify that _attribute_((constructor)) works as expected on a Power9 ppc64le machine from Beaker.

      I am not sure if this happens because ACVP parser does not correctly initialize NSS (which we could fix in ACVP parser), or whether this is a bug in NSS. If it is a bug in NSS, we would also have to address it in 9.0, which probably contains the same code.

      Version-Release number of selected component (if applicable):
      nss-3.67.0-7.el8_5

      How reproducible:
      Run ACVP tests with a test vector that generates a DH key.

      Actual results:
      DH key generation fails because the RNG is uninitialized

      Expected results:
      DH key generation succeeds.

            rrelyea@redhat.com Robert Relyea
            cllang@redhat.com Clemens Lang
            Robert Relyea Robert Relyea
            SSG Security QE SSG Security QE
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: