Notebook pod gets killed during spawning process

Description of problem:

In a test cluster we start facing the following error while spawning notebooks from KF Spawner page

It was happening for only one user

It was not happening from DS Projects feature (workbenches worked fine for the same user)

There was not a clear way for fixing it. After many spawning process, it got working again, but after few more tries the error came back.

Prerequisites (if any, like setup, operators/versions):

Steps to Reproduce

There isn't a reproducer for the issue, it just started happening on a cluster. These were the basic steps accomplished before facing the issue:

1. Log in Dashboard with a user 
2. launch Jupyter from "Enabled" page
3. spawn a notebook

Actual results:

user is not able to spawn notebooks

Expected results:

user is able to spawn notebooks

Reproducibility (Always/Intermittent/Only Once):

• Always in that test cluster with a single user (we didn't try all the possible users, though)

• Only in one cluster so far 

   

Build Details:

RHODS v1.22.0-2

Workaround:

TBD / N/A

Additional info:

Slack thread

I couldn't find relevant logs about what was happening. Next time we face it, we'll try to gather all the pod logs