Currently, if no assignment is found for either a group or user than a label is displayed. It can cause confusion if there are for instance users assigned but no groups. That cause the UI to display unassigned.