This is filed on behalf of a customer, who opened the case with the long quotation following this quick summary. The quotation is quite detailed and informative to their issue.

Essentially

• A cryptographic kernel module is hashed at load time. That hash is checked against a hard-coded build-time hash to attest for integrity.
• They are incurring difficulty where the KASLR and patching of machine code in the binary causes the computed hash to change across z-stream releases and even across different models of the same CPU vendor.
• They have outstanding questions on how to go about hashing their module's code and rodata segments that's agnostic of z-stream version and CPU

From the customer case;

Describe your problem. Include specific actions and error messages.
Executive Summary:

Zscaler is developing a FIPS certifiable product that leverages an external vendor’s (vendorX) Kernel mode FIPS-140-3 certifiable cryptographic kernel module.

During the development and integration of this feature we noticed the vendorX FIPS crypto module hash was very fragile and changed (needed re-building) when we moved between Kernel versions or installed the built crypto module on different Intel x86_64 processor models.

When we raised this hash stability issue with the crypto module VendorX, they indicated that RHEL kernels frequently backport fixes, improvements and security updates into the RHEL 9.6 Kernel. In their explanation this has a potential to de-stabilize the FIPS 140-3 crypto hash as the hash will be processor dependent due to the way RHEL 9.6 performs Kernel address relocations.

We are currently developing under RHEL 9.6 kernel-5.14.0-570.28.1.el9.x86_64, which appears to use a base 5.14.0 upstream Kernel with proprietary RHEL back ports denoted by 570.28.1

Zscaler wants to make our software and the vendorX FIPS 140-3 FIPS crypto module hash as hardware and kernel version independent as possible.

We are looking forward to any advice or best-practices guidance that RedHat may have in this matter so we can work with our Kernel crypto module vendorX to ensure FIPS 140-3 crypto hash stability across different x86_64 hardware platforms and RHEL Kernel versions.

Analysis of Crypto VendorX FIPS-140-3 Hashing:

Q1) What is the Vendor-X-SSL FIPS 140-3 Hash?
A1) The FIPS-140-3 integrity check for the VendorX kernel module is a Power-On Self-Test (POST) at Kernel module load time designed to verify the module's integrity at runtime. It computes an HMAC-SHA256 over the module's code and read-only data segments in memory. The kernel module on RHEL 9.6, is trying to overcome two major challenges: Address Space Layout Randomization (ASLR) and runtime code relocations.

The code in the VendorX Kernel module uses a mechanism that canonicalizes the module's executable code to reverse the effects of relocation before hashing, to try and ensure the runtime hash matches a pre-computed, embedded hash value.

Q2) How is the core hashing performed inside the VendorX Kernel module?

A2) Here’s a step-by-step breakdown of how hashing works for a kernel module:

The Hashing Algorithm

The hashing uses HMAC-SHA256 for the integrity check.
The key for the HMAC is a hardcoded hexadecimal string stored in the `coreKey` static variable.
static const char coreKey[] = "< … >";

Memory Regions to be Hashed

The hash is computed over two critical parts of the module's memory footprint:

• Executable Code Segment (`.text`): The module .text boundaries are defined by the symbols `vendorX_FIPS_first` and `vendorX_FIPS_last`
• Read-Only Data Segment (`.rodata`): The module .rodata boundaries are defined by `vendorX_FIPS_ro_start` and `vendorX_FIPS_ro_end`

These symbols are placed by a custom linker script during the module's build process to mark the exact start and end of the FIPS boundary.

Handling Kernel Address Space Layout Randomization (ASLR):

• On RHEL 9.6, kernel modules are loaded at a random base address for security. This means the addresses of `vendorX_FIPS_first` and `vendorX_FIPS_last` are not known at compile time.

• The code handles this using a redirection table when built for a Linux kernel module (`VENDORX_LINUXKM`) as a Position-Independent Executable (`_PIE_`).

#if defined(VENDORX_LINUXKM) && defined(__PIE__)
#define vendorX_FIPS_first vendorX_linuxkm_get_pie_redirect_table()->vendorX_FIPS_first
#define vendorX_FIPS_last vendorX_linuxkm_get_pie_redirect_table()->vendorX_FIPS_last
#else
int vendorX_FIPS_first(void);
int vendorX_FIPS_last(void);
#endif

This shows that instead of being fixed addresses, `vendorX_FIPS_first` and `vendorX_FIPS_last` are macros that call a function to look up the true, post-ASLR addresses from a table at runtime.

Code Canonicalization:

When the Linux kernel loader loads a module, it performs relocations, patching the machine code to fix up addresses of functions and variables. This means the in-memory version of the code is different from the on-disk version, which would cause a simple hash comparison to fail.

The vendorX solves this problem by "canonicalizing" the code segment before hashing it. This is controlled by the `VENDORX_TEXT_SEGMENT_CANONICALIZER` macro.

#ifdef VERNDORX_TEXT_SEGMENT_CANONICALIZER
    {
        // ... setup buffer ...
        const byte *text_p = (const byte *)first;
        while (text_p < (const byte *)last) {
            ssize_t progress =
                VENDORX_TEXT_SEGMENT_CANONICALIZER(
                    text_p,
                    min(8192, (word32)((const byte *)last - text_p)),
                    buf, &cur_reloc_index);
            // ...
            ret = vendorx_HmacUpdate_fips(&hmac, buf, (word32)progress);
            // ...
        }
        // ...
    }
#else /* !VENDORX_TEXT_SEGMENT_CANONICALIZER */
    ret = vendorx_HmacUpdate_fips(&hmac, (byte *)(wc_ptr_t)first, (word32)code_sz);
#endif

Instead of hashing the code segment directly, the code iterates through it and passes chunks to the `VENDORX_TEXT_SEGMENT_CANONICALIZER` function, it is responsible for:

Reading a chunk of the in-memory, relocated code.
Identifying any bytes that were modified by the kernel loader.
Replacing these modified bytes with their original, "canonical" values from the pre-relocation `.ko` file.
Writing the resulting "clean" code into a temporary buffer (`buf`).

It is this canonicalized code in `buf` that is then passed to `vendorx_HmacUpdate_fips`. This process effectively reverses the relocations for the purpose of the hash calculation.

Final Hash and Verification:

1. After the canonicalized code segment is hashed, the read-only data segment is hashed directly (as it typically does not contain relocations).
2. `vendorX_HmacFinal_fips` is called to produce the final HMAC-SHA256 digest.
3. This computed hash is compared against a pre-calculated, hardcoded hash stored in the `verifyCore` variable.

#ifndef VENDORX_FIPS_CORE_DYNAMIC_HASH_VALUE
static const
#endif
char verifyCore[] =
#ifdef VENDORX_FIPS_CORE_HASH_VALUE
WC_STRINGIFY(VENDORX_FIPS_CORE_HASH_VALUE);
#else
"B2DBD34A8A411B028587FE550699FCEACA213CACC378E39F1AD644B6DD53BDE2";
#endif

This `verifyCore` value is generated at build time by performing the exact same canonicalization and HMAC-SHA256 process on the static `.ko` module file.

If the runtime hash matches the build-time hash, the module's integrity is verified, and the POST succeeds.

Summary

• The FIPS hash computation for the VendorX kernel module under RHEL 9.6 integrity check specifically deals with the kernel environment.
• It handles dynamic module loading addresses (ASLR) and reverses the effects of runtime code relocations through a canonicalization process before hashing.
• It verifies runtime in-memory hash against a hash generated at build time, fulfilling a core requirement of FIPS 140 validation.

Open Questions for RedHat:

Question #1: How can we make the FIPS-140-3 hash not change within the same RedHat release i.e RHEL 9.6 stream or RHEL 9.7 stream etc during minor Kernel upgrades?
Question #2: How can we ensure the FIPS hash stays stable across a given Intel x86_64 processor family (e.g. Intel Xeon Gold Gen 5, Xeon Silver Gen 4 etc)?

• Reference: Intel Processor Families
• Reference: Intel Xeon Gen 5 Processors
  Question #3: Any additional RedHat recommended best practices for FIPS 140-3 crypto modules under RHEL 9.6 Kernels?

Describe the impact to you or the business

• We are unable to ship a single binary of our FIPS kernel module that will maintain a stable FIPS hash across different Intel x86_64 platforms that are used by our customers with RHEL 9.6
• Even minor kernel version upgrades within RHEL 9.6 may require us to re-build the FIPS 140-3 Kernel module

In what environment are you experiencing this behavior?
We have seen this issue with Kernel 5.14.0-570.28.1.el9_6 and other minor upgrades from this Kernel