Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-58764

switching to a bootable container image causes sshd to fail to start due to host key permissions

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Normal Normal
    • None
    • rhel-9.4
    • rhel-bootc-container
    • None
    • No
    • None
    • sst_bootc
    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • None

      What were you trying to do that didn't work?

      I was switching a RHEL for Edge system to use a bootable container image (i.e. Bifrost/image mode for RHEL) and when it rebooted into the new container image, sshd failed to start due to host key permissions.

      What is the impact of this issue to you?

      This prevents users from migrating from an ostree commit based RHEL for Edge model to the image mode for RHEL model.

      Please provide the package NVR for which the bug is seen:

      openssh-server-8.7p1-38.el9_4.4.x86_64

      How reproducible is this bug?:

      Always

      Steps to reproduce

      1. Create a blueprint that includes the bootc RPM
      2. Use osbuild to produce an edge-commit and edge-installer artifact
      3. Deploy the edge-commit (via edge-installer) to a system/VM
      4. Create a Containerfile that includes packages found in RHEL for Edge (https://gitlab.cee.redhat.com/miabbott/rhel-for-edge-bootc-spike/-/blob/main/Containerfile?ref_type=heads)
      5. Build the container image and push to a registry
      6. Use `bootc switch` to switch to the container image
      7. Reboot

      Expected results

      System boots normally; sshd starts successfully

      Actual results

      $ systemctl status sshd.service --no-pager
      ● sshd.service - OpenSSH server daemon
           Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; preset: enabled)
           Active: activating (auto-restart) (Result: exit-code) since Thu 2024-09-12 21:51:04 UTC; 33s ago
             Docs: man:sshd(8)
                   man:sshd_config(5)
          Process: 857 ExecStart=/usr/sbin/sshd -D $OPTIONS (code=exited, status=1/FAILURE)
         Main PID: 857 (code=exited, status=1/FAILURE)
              CPU: 16ms
      [core@localhost ~]$ sudo journalctl -b -u sshd --no-pager
      Sep 12 21:51:04 localhost.localdomain systemd[1]: Starting OpenSSH server daemon...
      Sep 12 21:51:04 localhost.localdomain sshd[857]: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
      Sep 12 21:51:04 localhost.localdomain sshd[857]: @         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
      Sep 12 21:51:04 localhost.localdomain sshd[857]: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
      Sep 12 21:51:04 localhost.localdomain sshd[857]: Permissions 0640 for '/etc/ssh/ssh_host_rsa_key' are too open.
      Sep 12 21:51:04 localhost.localdomain sshd[857]: It is required that your private key files are NOT accessible by others.
      Sep 12 21:51:04 localhost.localdomain sshd[857]: This private key will be ignored.
      Sep 12 21:51:04 localhost.localdomain sshd[857]: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
      Sep 12 21:51:04 localhost.localdomain sshd[857]: @         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
      Sep 12 21:51:04 localhost.localdomain sshd[857]: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
      Sep 12 21:51:04 localhost.localdomain sshd[857]: Permissions 0640 for '/etc/ssh/ssh_host_ecdsa_key' are too open.
      Sep 12 21:51:04 localhost.localdomain sshd[857]: It is required that your private key files are NOT accessible by others.
      Sep 12 21:51:04 localhost.localdomain sshd[857]: This private key will be ignored.
      Sep 12 21:51:04 localhost.localdomain sshd[857]: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
      Sep 12 21:51:04 localhost.localdomain sshd[857]: @         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
      Sep 12 21:51:04 localhost.localdomain sshd[857]: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
      Sep 12 21:51:04 localhost.localdomain sshd[857]: Permissions 0640 for '/etc/ssh/ssh_host_ed25519_key' are too open.
      Sep 12 21:51:04 localhost.localdomain sshd[857]: It is required that your private key files are NOT accessible by others.
      Sep 12 21:51:04 localhost.localdomain sshd[857]: This private key will be ignored.
      Sep 12 21:51:04 localhost.localdomain sshd[857]: sshd: no hostkeys available -- exiting.
      Sep 12 21:51:04 localhost.localdomain systemd[1]: sshd.service: Main process exited, code=exited, status=1/FAILURE
      Sep 12 21:51:04 localhost.localdomain systemd[1]: sshd.service: Failed with result 'exit-code'.
      Sep 12 21:51:04 localhost.localdomain systemd[1]: Failed to start OpenSSH server daemon.
      Sep 12 21:51:47 localhost.localdomain systemd[1]: sshd.service: Scheduled restart job, restart counter is at 1.
      Sep 12 21:51:47 localhost.localdomain systemd[1]: Stopped OpenSSH server daemon.
      Sep 12 21:51:47 localhost.localdomain systemd[1]: Starting OpenSSH server daemon...
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: @         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: Permissions 0640 for '/etc/ssh/ssh_host_rsa_key' are too open.
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: It is required that your private key files are NOT accessible by others.
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: This private key will be ignored.
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: @         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: Permissions 0640 for '/etc/ssh/ssh_host_ecdsa_key' are too open.
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: It is required that your private key files are NOT accessible by others.
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: This private key will be ignored.
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: @         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: Permissions 0640 for '/etc/ssh/ssh_host_ed25519_key' are too open.
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: It is required that your private key files are NOT accessible by others.
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: This private key will be ignored.
      Sep 12 21:51:47 localhost.localdomain sshd[4571]: sshd: no hostkeys available -- exiting.
      Sep 12 21:51:47 localhost.localdomain systemd[1]: sshd.service: Main process exited, code=exited, status=1/FAILURE
      Sep 12 21:51:47 localhost.localdomain systemd[1]: sshd.service: Failed with result 'exit-code'.
      Sep 12 21:51:47 localhost.localdomain systemd[1]: Failed to start OpenSSH server daemon.
      

      Extra Info

      This may be related to https://fedoraproject.org/wiki/Changes/SSHKeySignSuidBit

      There was a service added to the Fedora openssh-server RPM to handle this change: https://src.fedoraproject.org/rpms/openssh/pull-request/40

      But it doesn't seem to be present in the RHEL version of openssh-server
       

            walters@redhat.com Colin Walters
            miabbott@redhat.com Micah Abbott
            Colin Walters Colin Walters
            Wei Shi Wei Shi
            Gabriela Necasova Gabriela Necasova
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: