Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-28691

Using JDK Flight Recorder on Tomcat service lead to various disallowing access.

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • rhel-8.9.0
    • tomcat
    • None
    • Important
    • rhel-sst-middleware
    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • Red Hat Enterprise Linux
    • None
    • None
    • None
    • None

      What were you trying to do that didn't work?

      JDK Flight Recorder unable to make dump on tomcat service in 'Enforcing' mode.

      Please provide the package NVR for which bug is seen:

      See the reproducting steps.

      How reproducible:

      Always.

      Steps to reproduce

      1. Update the selinux, tomcat, java, kernel packages to the latest version for RHEL8.

      1. rpm -qa | egrep 'selinux|tomcat|kernel|jdk' | sort

      container-selinux-2.167.0-1.module+el8.5.0+12582+56d94c81.noarch
      copy-jdk-configs-4.0-2.el8.noarch
      flatpak-selinux-1.8.5-4.el8.noarch
      java-17-openjdk-17.0.10.0.7-2.el8.x86_64
      java-17-openjdk-devel-17.0.10.0.7-2.el8.x86_64
      java-17-openjdk-headless-17.0.10.0.7-2.el8.x86_64
      java-1.8.0-openjdk-headless-1.8.0.402.b06-2.el8.x86_64
      kernel-4.18.0-348.el8.x86_64
      kernel-4.18.0-513.18.1.el8_9.x86_64
      kernel-core-4.18.0-348.el8.x86_64
      kernel-core-4.18.0-513.18.1.el8_9.x86_64
      kernel-modules-4.18.0-348.el8.x86_64
      kernel-modules-4.18.0-513.18.1.el8_9.x86_64
      kernel-tools-4.18.0-348.el8.x86_64
      kernel-tools-libs-4.18.0-348.el8.x86_64
      libselinux-2.9-5.el8.x86_64
      libselinux-utils-2.9-5.el8.x86_64
      python3-libselinux-2.9-5.el8.x86_64
      rpm-plugin-selinux-4.14.3-19.el8.x86_64
      selinux-policy-3.14.3-128.el8_9.1.noarch
      selinux-policy-devel-3.14.3-128.el8_9.1.noarch
      selinux-policy-doc-3.14.3-128.el8_9.1.noarch
      selinux-policy-targeted-3.14.3-128.el8_9.1.noarch
      tomcat-9.0.62-27.el8_9.3.noarch
      tomcat-el-3.0-api-9.0.62-27.el8_9.3.noarch
      tomcat-jsp-2.3-api-9.0.62-27.el8_9.3.noarch
      tomcat-lib-9.0.62-27.el8_9.3.noarch
      tomcat-servlet-4.0-api-9.0.62-27.el8_9.3.noarch

      1. uname -a
        Linux rhel8u5gui.example.com 4.18.0-513.18.1.el8_9.x86_64 #1 SMP Thu Feb 1 03:51:05 EST 2024 x86_64 x86_64 x86_64 GNU/Linux

      2. Start the system in GUI mode.

      1. ps aux | grep pulsea
        gdm 5732 0.2 0.8 876444 14588 ? Ssl 11:58 0:00 /usr/bin/pulseaudio --daemonize=no --log-target=journal
        root 6100 0.0 0.0 12136 1092 pts/0 S+ 11:59 0:00 grep --color=auto pulsea

      3. Ensure your system is running in 'Enforcing' mode at the beginning.

      1. getenforce
        Enforcing

      4. Remove all old logs for the audit and only keep the logs for current boot.

      1. ls -la /var/log/audit/audit.log
        rw------. 1 root root 99332 Mar 9 11:59 /var/log/audit/audit.log

      5. Restart the tomcat to make sure it's a newly started one. Mark down its main PID.

      1. systemctl stop tomcat
      1. systemctl start tomcat
      1. systemctl status tomcat
        ● tomcat.service - Apache Tomcat Web Application Container
        Loaded: loaded (/usr/lib/systemd/system/tomcat.service; disabled; vendor preset: disabled)
        Active: active (running) since Sat 2024-03-09 12:00:55 HKT; 5s ago
        Main PID: 6128 (java)
        Tasks: 27 (limit: 10939)
        Memory: 155.3M
        CGroup: /system.slice/tomcat.service
        └─6128 /usr/lib/jvm/jre/bin/java -Djavax.sql.DataSource.Factory=org.apache.commons.dbcp.BasicDataSourceFactory -classpath /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bi>

      Mar 09 12:00:57 rhel8u5gui.example.com server[6128]: 09-Mar-2024 12:00:57.960 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/var/>
      Mar 09 12:00:57 rhel8u5gui.example.com server[6128]: 09-Mar-2024 12:00:57.961 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.co>
      Mar 09 12:00:57 rhel8u5gui.example.com server[6128]: 09-Mar-2024 12:00:57.961 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.ma>
      Mar 09 12:00:57 rhel8u5gui.example.com server[6128]: 09-Mar-2024 12:00:57.969 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent The Apache Tomcat Native library which>
      Mar 09 12:00:59 rhel8u5gui.example.com server[6128]: 09-Mar-2024 12:00:59.231 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"]
      Mar 09 12:00:59 rhel8u5gui.example.com server[6128]: 09-Mar-2024 12:00:59.312 INFO [main] org.apache.catalina.startup.Catalina.load Server initialization in [2115] milliseconds
      Mar 09 12:00:59 rhel8u5gui.example.com server[6128]: 09-Mar-2024 12:00:59.386 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina]
      Mar 09 12:00:59 rhel8u5gui.example.com server[6128]: 09-Mar-2024 12:00:59.387 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet engine: [Apache Tomcat/9.0.6>
      Mar 09 12:00:59 rhel8u5gui.example.com server[6128]: 09-Mar-2024 12:00:59.428 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
      Mar 09 12:00:59 rhel8u5gui.example.com server[6128]: 09-Mar-2024 12:00:59.494 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in [181] milliseconds

      6. No AVC entries in current audit log.

      1. ausearch -i -m AVC,USER_AVC
        <no matches>

      7. Switch the system to permissive mode. And you'll see the two AVC entries.

      1. setenforce 0
      1. ausearch -i -m AVC,USER_AVC

        type=USER_AVC msg=audit(03/09/2024 12:01:36.274:184) : pid=726 uid=dbus auid=unset ses=unset subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: received setenforce notice (enforcing=0) exe=/usr/bin/dbus-daemon sauid=dbus hostname=? addr=? terminal=?'


        type=USER_AVC msg=audit(03/09/2024 12:01:36.274:185) : pid=4691 uid=root auid=root ses=3 subj=unconfined_u:unconfined_r:unconfined_dbusd_t:s0-s0:c0.c1023 msg='avc: received setenforce notice (enforcing=0) exe=/usr/bin/dbus-daemon sauid=root hostname=? addr=? terminal=?'

      8. Check the PIDs again.

      1. ps auxwww | grep -e tomcat -e pulseaudio
        gdm 5732 0.0 0.8 876444 14588 ? Ssl 11:58 0:00 /usr/bin/pulseaudio --daemonize=no --log-target=journal
        tomcat 6128 2.2 5.1 2391284 92368 ? Ssl 12:00 0:02 /usr/lib/jvm/jre/bin/java -Djavax.sql.DataSource.Factory=org.apache.commons.dbcp.BasicDataSourceFactory -classpath /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar: -Dcatalina.base=/usr/share/tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/usr/share/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager org.apache.catalina.startup.Bootstrap start
        root 6199 0.0 0.0 12136 1128 pts/0 R+ 12:02 0:00 grep --color=auto -e tomcat -e pulseaudio

      9. Is using default tomcat user.

      1. id tomcat
        uid=53(tomcat) gid=53(tomcat) groups=53(tomcat)

      10. Start a JFR.start on that tomcat process(6128).

      1. su tomcat --shell=/bin/bash -c "/usr/lib/jvm/java-17-openjdk/bin/jcmd 6128 JFR.start"
        6128:
        Started recording 1. No limit specified, using maxsize=250MB as default.

      Use jcmd 6128 JFR.dump name=1 filename=FILEPATH to copy recording data to file.

      11. Start JFR.dump on that process 6128.

      1. su tomcat --shell=/bin/bash -c "/usr/lib/jvm/java-17-openjdk/bin/jcmd 6128 JFR.dump filename=/tmp/testdump.jfr"
        6128:
        Dumped recording, 348.4 kB written to:

      /tmp/testdump.jfr

      12. Lots of AVC entries show and generate suggestions for the denied operations.

      1. cat /var/log/audit/audit.log| audit2allow

      #============= tomcat_t ==============
      allow tomcat_t NetworkManager_t:dir

      { getattr search };
      allow tomcat_t NetworkManager_t:file { getattr open read };
      allow tomcat_t NetworkManager_t:lnk_file { getattr read };
      allow tomcat_t accountsd_t:dir { getattr search }

      ;
      allow tomcat_t accountsd_t:file
      { getattr open read };
      allow tomcat_t accountsd_t:lnk_file { getattr read };
      allow tomcat_t alsa_t:dir { getattr search };
      allow tomcat_t alsa_t:file { getattr open read }

      ;
      allow tomcat_t alsa_t:lnk_file
      { getattr read };
      allow tomcat_t auditd_t:dir { getattr search };
      allow tomcat_t auditd_t:file { getattr open read };
      allow tomcat_t auditd_t:lnk_file { getattr read }

      ;
      allow tomcat_t avahi_t:dir
      { getattr search }

      ;
      allow tomcat_t avahi_t:file { getattr open read };
      allow tomcat_t avahi_t:lnk_file { getattr read };
      allow tomcat_t chronyd_t:dir { getattr search }

      ;
      allow tomcat_t chronyd_t:file

      { getattr open read };
      allow tomcat_t chronyd_t:lnk_file { getattr read };
      allow tomcat_t crond_t:dir { getattr search };
      allow tomcat_t crond_t:file { getattr open read }

      ;
      allow tomcat_t crond_t:lnk_file
      { getattr read };
      allow tomcat_t cupsd_t:dir { getattr search };
      allow tomcat_t cupsd_t:file { getattr open read };
      allow tomcat_t cupsd_t:lnk_file { getattr read }

      ;
      allow tomcat_t devicekit_disk_t:dir
      { getattr search };
      allow tomcat_t devicekit_disk_t:file { getattr open read };
      allow tomcat_t devicekit_disk_t:lnk_file { getattr read };
      allow tomcat_t firewalld_t:dir { getattr search }

      ;
      allow tomcat_t firewalld_t:file
      { getattr open read }

      ;
      allow tomcat_t firewalld_t:lnk_file { getattr read };
      allow tomcat_t fsdaemon_t:dir { getattr search };
      allow tomcat_t fsdaemon_t:file { getattr open read }

      ;
      allow tomcat_t fsdaemon_t:lnk_file

      { getattr read };
      allow tomcat_t gssproxy_t:dir { getattr search };
      allow tomcat_t gssproxy_t:file { getattr open read };
      allow tomcat_t gssproxy_t:lnk_file { getattr read }

      ;
      allow tomcat_t init_t:file
      { getattr open read };
      allow tomcat_t init_t:lnk_file { getattr read };
      allow tomcat_t kernel_t:dir { getattr search };
      allow tomcat_t kernel_t:file { getattr open read }

      ;
      allow tomcat_t kernel_t:lnk_file
      { getattr read }

      ;
      allow tomcat_t ksmtuned_t:dir { getattr search };
      allow tomcat_t ksmtuned_t:file { getattr open read };
      allow tomcat_t ksmtuned_t:lnk_file { getattr read }

      ;
      allow tomcat_t lsmd_t:dir

      { getattr search };
      allow tomcat_t lsmd_t:file { getattr open read };
      allow tomcat_t lsmd_t:lnk_file { getattr read };
      allow tomcat_t mcelog_t:dir { getattr search }

      ;
      allow tomcat_t mcelog_t:file
      { getattr open read };
      allow tomcat_t mcelog_t:lnk_file { getattr read };
      allow tomcat_t modemmanager_t:dir { getattr search };
      allow tomcat_t modemmanager_t:file { getattr open read }

      ;
      allow tomcat_t modemmanager_t:lnk_file
      { getattr read };
      allow tomcat_t policykit_t:dir { getattr search };
      allow tomcat_t policykit_t:file { getattr open read };
      allow tomcat_t policykit_t:lnk_file { getattr read }

      ;
      allow tomcat_t rhsmcertd_t:dir
      { getattr search }

      ;
      allow tomcat_t rhsmcertd_t:file { getattr open read };
      allow tomcat_t rhsmcertd_t:lnk_file { getattr read };
      allow tomcat_t rpcbind_t:dir { getattr search }

      ;
      allow tomcat_t rpcbind_t:file

      { getattr open read };
      allow tomcat_t rpcbind_t:lnk_file { getattr read };
      allow tomcat_t rtkit_daemon_t:dir { getattr search };
      allow tomcat_t rtkit_daemon_t:file { getattr open read }

      ;
      allow tomcat_t rtkit_daemon_t:lnk_file
      { getattr read };
      allow tomcat_t sshd_t:dir { getattr search };
      allow tomcat_t sshd_t:file { getattr open read };
      allow tomcat_t sshd_t:lnk_file { getattr read }

      ;
      allow tomcat_t sssd_t:dir
      { getattr search };
      allow tomcat_t sssd_t:file { getattr open read };
      allow tomcat_t sssd_t:lnk_file { getattr read };
      allow tomcat_t syslogd_t:dir { getattr search }

      ;
      allow tomcat_t syslogd_t:file
      { getattr open read }

      ;
      allow tomcat_t syslogd_t:lnk_file { getattr read };
      allow tomcat_t system_dbusd_t:dir { getattr search };
      allow tomcat_t system_dbusd_t:file { getattr open read }

      ;
      allow tomcat_t system_dbusd_t:lnk_file

      { getattr read };
      allow tomcat_t systemd_logind_t:dir { getattr search };
      allow tomcat_t systemd_logind_t:file { getattr open read };
      allow tomcat_t systemd_logind_t:lnk_file { getattr read }

      ;
      allow tomcat_t systemd_machined_t:dir
      { getattr search };
      allow tomcat_t systemd_machined_t:file { getattr open read };
      allow tomcat_t systemd_machined_t:lnk_file { getattr read };
      allow tomcat_t tmp_t:sock_file { create rename setattr };
      allow tomcat_t tuned_t:dir { getattr search }

      ;
      allow tomcat_t tuned_t:file
      { getattr open read };
      allow tomcat_t tuned_t:lnk_file { getattr read };
      allow tomcat_t udev_t:dir { getattr search };
      allow tomcat_t udev_t:file { getattr open read }

      ;
      allow tomcat_t udev_t:lnk_file
      { getattr read }

      ;
      allow tomcat_t unconfined_dbusd_t:dir { getattr search };
      allow tomcat_t unconfined_dbusd_t:file { getattr open read };
      allow tomcat_t unconfined_dbusd_t:lnk_file getattr;
      allow tomcat_t unconfined_t:dir { getattr search };
      allow tomcat_t unconfined_t:file { getattr open read };
      allow tomcat_t unconfined_t:lnk_file { getattr read }

      ;
      allow tomcat_t xdm_t:dir

      { getattr search }

      ;
      allow tomcat_t xdm_t:file

      { getattr open read }

      ;
      allow tomcat_t xdm_t:lnk_file

      { getattr read }

      ;

      13. Stop auditd to keep the audit.log.

      1. service auditd stop
        Stopping logging: [ OK ]

      14. Attach the log to this JIRA.

      Expected results

      JDK Flight Recorder should be able to make dump on tomcat service in 'Enforcing' mode.

      Actual results

      JDK Flight Recorder is unable to make dump on tomcat service in 'Enforcing' mode.

              akrajcik@redhat.com Adam Krajcik
              rhn-support-qguo Qi Guo
              Hui Wang Hui Wang (Inactive)
              no-user-match-found no-user-match-found
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated: