Description of problem:

On RHEL 7 and later, it is possible to use PAM authentication by following
the instructions in solution 7028399:

How to use PAM authentication with Virtual Network Computing (VNC) on
Red Hat Enterprise Linux
https://access.redhat.com/solutions/7028399

On RHEL 7 (tigervnc 1.8.0), it is possible to enable PAM only for the user who
owns the Xvnc process by adding this to /etc/tigervnc/vncserver-config-defaults:

SecurityTypes="TLSPlain"
PlainUsers="$USER"

On RHEL 8 and later, however, this does not work.

Version-Release number of selected component (if applicable):

tigervnc-server-1.12.0-15.el8_8.x86_64

How reproducible:

Always

Steps to Reproduce:

1. Follow the instructions to configure the VNC service but skip the step to
set a VNC password for the user:

• For RHEL 7, refer to https://access.redhat.com/solutions/966063
• For RHEL 8.3 and later, refer to section 5.3 of the Using the desktop
  environment in RHEL 8 guide.
• For RHEL 9, refer to chapter 9 of the Getting started with the GNOME
  desktop environment guide.

2. Ensure that a vnc PAM service exists:

1. test -f /etc/pam.d/vnc || ln -s login /etc/pam.d/vnc

3. Enable PAM authentication only for the user who is running Xvnc by adding
these lines to /etc/tigervnc/vncserver-config-defaults:

SecurityTypes="TLSPlain"
PlainUsers="$USER"

Notice that on RHEL 7 it is necessary to create the /etc/tigervnc/
directory first, since i does not exist by default:

1. mkdir -p /etc/tigervnc/

4. Start the VNC service:

1. systemctl start vncserver@:1.service

5. Check if the configuration worked:

1. ps aux| fgrep Xvnc | grep -v -e grep -e xinit

Actual results:

vncuser1 7398 0.1 1.9 284652 72912 ? S 13:56 0:00 /usr/bin/Xvnc :1 -plainusers "$USER" -securitytypes "TLSPlain" -auth /home/vncuser1/.Xauthority -desktop rhel-8-2.example.com:1 (vncuser1) -fp catalogue:/etc/X11/fontpath.d -pn -rfbauth /home/vncuser1/.vnc/passwd -rfbport 5901

Expected results:

Same result seen on RHEL 7:

vncuser1 7688 0.3 1.9 218336 36252 ? Sl 13:57 0:00 /usr/bin/Xvnc :1 -plainusers vncuser1 -securitytypes TLSPlain -auth /home/vncuser1/.Xauthority -desktop rhel-7-2.example.com:1 (vncuser1) -fp catalogue:/etc/X11/fontpath.d -geometry 1024x768 -pn -rfbauth /home/vncuser1/.vnc/passwd -rfbport 5901 -rfbwait 30000

Additional info:

The problem happens because on RHEL 7 the service uses the "vncserver" script,
which expands environment variables in the configuration files, while on RHEL 8
the service uses "vncsession", which does not recognize environment variables
in configuration files.

A simple workaround on RHEL 8 is to use

PlainUsers=*

but this creates a security vulnerability, since it allows any user to have
access to the VNC session, not only the owner of the "Xvnc" process.

Another workaround is to create a ~/.vnc/config containing

PlainUsers=<actual-user-name>

but this is inconvenient, since it requires the system administrator to log
in as each vnc user to create the configuration file.