Description of problem:

When modifying the pod toleration settings (or enabling it), new notebooks being spawned won't pick up the change for the first 5/10 seconds.

It's hard to pin down an exact number, but I've seen it fail to pick up the change at exactly 5s from the setting being changed and pick it up correctly at exactly 10s from the settings being changed.

It is possible other settings behave similarly but I have not confirmed it yet

Prerequisites (if any, like setup, operators/versions):

latest RHODS 1.16 RC

Steps to Reproduce

1. Go to cluster settings page
2. Modify the setting for pod tolerations
3. Save changes
4. Spawn a notebook within 5 to 10 seconds from the settings being changed

Actual results:

If the notebook was spawned too quickly, the change in setting won't be picked up

Expected results:

Once the setting is saved and the user receives a positive feedback I'd expect the change to always apply.

Reproducibility (Always/Intermittent/Only Once):

always

Build Details:

Workaround:

Additional info: