Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-52491

NSS rebase breaks systemtap's ability to grant trust to a compile server

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • rhel-10.0.beta
    • rhel-10.0.beta
    • systemtap
    • None
    • systemtap-5.1-9.el10
    • No
    • None
    • Rebase
    • 1
    • rhel-sst-pt-perf-debug
    • ssg_platform_tools
    • 25
    • 25
    • 2
    • False
    • Hide

      None

      Show
      None
    • No
    • Perf Debug Sprint 8
    • Unspecified Release Note Type - Unknown
    • None

      Let me try to rephrase RHEL-50692 so that it's short and to the point.  Between RHEL-10.0-20240709.23 and RHEL-10.0-20240801.0 there was a NSS rebase.  It broke systemtap's ability to grant trust to a compile server.

      Below, I'm testing this using currently latest systemtap-5.1-8.el10 in old/good versus new/broken environment.  The latest "good" nss version was nss-3.97.0-2.el10.  First "bad" nss version is nss-3.101.0-1.el10.

      The following example shows how after installing systemtap-5.1-8.el10 on top of old RHEL-10.0-20240709.23 and starting stap-server, local stap client can grant trust to the local server.

       

      
      RHEL-10.0-20240709.23  (+ additional repo for systemtap-5.1-8.el10)
      -------------------------------------------------------------------------------------------
      el10 x86_64 # yum install systemtap{,-server} net-tools
      el10 x86_64 # rpm -qa | grep -e systemtap -e ^nss | sort
      nss-3.97.0-1.el10.x86_64
      nss-softokn-3.97.0-1.el10.x86_64
      nss-softokn-freebl-3.97.0-1.el10.x86_64
      nss-sysinit-3.97.0-1.el10.x86_64
      nss-util-3.97.0-1.el10.x86_64
      systemtap-5.1-8.el10.x86_64
      systemtap-client-5.1-8.el10.x86_64
      systemtap-devel-5.1-8.el10.x86_64
      systemtap-runtime-5.1-8.el10.x86_64
      systemtap-server-5.1-8.el10.x86_64
      el10 x86_64 # 
      el10 x86_64 # service stap-server start
      Redirecting to /bin/systemctl start stap-server.service
      el10 x86_64 # SERVER_PORT=$( netstat -tlpn | awk '/stap/ {print $4}' | grep -o '[0-9]*$' ); SERVER_IP=127.0.0.1; echo $SERVER_IP:$SERVER_PORT
      127.0.0.1:44937
      el10 x86_64 # 
      el10 x86_64 # stap --trust-servers=ssl,signer,all-users,no-prompt --use-server=$SERVER_IP:$SERVER_PORT
      Adding trust in the following servers as an SSL peer for all users and as a module signer for all users
         host=unknown address=127.0.0.1 port=44937 sysinfo="unknown" version=unknown certinfo="unknown"
      el10 x86_64 #
      el10 x86_64 # certutil -d sql:/etc/systemtap/ssl/client -L
      Certificate Nickname                                         Trust Attributes
                                                                   SSL,S/MIME,JAR/XPI
      stap-server                                                  P,P,P
      el10 x86_64 # certutil -d sql:/etc/systemtap/staprun -L
      Certificate Nickname                                         Trust Attributes
                                                                   SSL,S/MIME,JAR/XPI
      stap-server                                                  P,P,P
      el10 x86_64 # 
      

       

      The following example shows how after installing systemtap-5.1-8.el10 on top of new RHEL-10.0-20240801.0  and starting stap-server, local stap client can NOT grant trust to the local server.

       

      RHEL-10.0-20240801.0  (having systemtap-5.1-8.el10)
      -------------------------------------------------------------------------------------------
      el10 x86_64 # service stap-server start
      Redirecting to /bin/systemctl start stap-server.service
      el10 x86_64 # rpm -qa | grep -e systemtap -e ^nss | sort
      nss-3.101.0-5.el10.x86_64
      nss-softokn-3.101.0-5.el10.x86_64
      nss-softokn-freebl-3.101.0-5.el10.x86_64
      nss-sysinit-3.101.0-5.el10.x86_64
      nss-util-3.101.0-5.el10.x86_64
      systemtap-5.1-8.el10.x86_64
      systemtap-client-5.1-8.el10.x86_64
      systemtap-devel-5.1-8.el10.x86_64
      systemtap-runtime-5.1-8.el10.x86_64
      systemtap-server-5.1-8.el10.x86_64
      el10 x86_64 # 
      el10 x86_64 # SERVER_PORT=$( netstat -tlpn | awk '/stap/ {print $4}' | grep -o '[0-9]*$' ); SERVER_IP=127.0.0.1; echo $SERVER_IP:$SERVER_PORT
      127.0.0.1:40895
      el10 x86_64 # 
      el10 x86_64 # stap --trust-servers=ssl,signer,all-users,no-prompt --use-server=$SERVER_IP:$SERVER_PORT
      Adding trust in the following servers as an SSL peer for all users and as a module signer for all users
         host=unknown address=127.0.0.1 port=40895 sysinfo="unknown" version=unknown certinfo="unknown"
      Unable to connect to host=unknown address=127.0.0.1 port=40895 sysinfo="unknown" version=unknown certinfo="unknown"
      (-8179 SEC_ERROR_UNKNOWN_ISSUER) Peer's Certificate issuer is not recognized.
      Unable to connect to host=unknown address=127.0.0.1 port=40895 sysinfo="unknown" version=unknown certinfo="unknown"
      (-8179 SEC_ERROR_UNKNOWN_ISSUER) Peer's Certificate issuer is not recognized.
      el10 x86_64 #
      el10 x86_64 # /usr/libexec/systemtap/stap-authorize-cert /var/lib/stap-server/.systemtap/ssl/server/stap.cert /etc/systemtap/ssl/client
      el10 x86_64 # /usr/libexec/systemtap/stap-authorize-cert /var/lib/stap-server/.systemtap/ssl/server/stap.cert /etc/systemtap/staprun
      el10 x86_64 # 
      el10 x86_64 # stap --trust-servers=ssl,signer,all-users,no-prompt --use-server=$SERVER_IP:$SERVER_PORT
      Adding trust in the following servers as an SSL peer for all users and as a module signer for all users
         host=unknown address=127.0.0.1 port=40895 sysinfo="unknown" version=unknown certinfo="unknown"
      el10 x86_64 #
      
      el10 x86_64 # certutil -d sql:/etc/systemtap/ssl/client -L
      Database needs user init
      Certificate Nickname                                         Trust Attributes
                                                                   SSL,S/MIME,JAR/XPI
      el10 x86_64 # certutil -d sql:/etc/systemtap/staprun -L
      Database needs user init
      Certificate Nickname                                         Trust Attributes
                                                                   SSL,S/MIME,JAR/XPI
      el10 x86_64 # 
      

       

       

      As the above experiment shows, with rebased NSS, systemtap client creates broken certificate database files which, as certutile shows, "need user init".

       

      This problem can be worked around by calling stap-authorize-cert:

      /usr/libexec/systemtap/stap-authorize-cert /var/lib/stap-server/.systemtap/ssl/server/stap.cert /etc/systemtap/ssl/client
      /usr/libexec/systemtap/stap-authorize-cert /var/lib/stap-server/.systemtap/ssl/server/stap.cert /etc/systemtap/staprun

      After this, broken things start working again.

              rhtorsys Frank Eigler
              mcermak@redhat.com Martin Cermak
              Frank Eigler Frank Eigler
              Martin Cermak Martin Cermak
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated: