Description of problem:

Duplicate pods are getting created on a node with similar names but with the different IPs. 

~~~

$ cat crio/crictl_ps | grep -i pcrf-gtre-1
4a0dca756fb09       90204dcd292cab936b303f2194f4613a8dd97adef70be699742934b83cfe65a5                                                                             3 days ago          Running             pcrf-gtre                            0                   4ce881fa49686       pcrf-gtre-1
d1c96f3bca86e       90204dcd292cab936b303f2194f4613a8dd97adef70be699742934b83cfe65a5                                                                             3 days ago          Running             pcrf-gtre                            0                   5b4f3e54f955a       pcrf-gtre-1

$ cat crio/crictl_ps_-a | grep -i pcrf-gtre-1
4a0dca756fb09       90204dcd292cab936b303f2194f4613a8dd97adef70be699742934b83cfe65a5                                                                             3 days ago          Running             pcrf-gtre                            0                   4ce881fa49686       pcrf-gtre-1
d1c96f3bca86e       90204dcd292cab936b303f2194f4613a8dd97adef70be699742934b83cfe65a5                                                                             3 days ago          Running             pcrf-gtre                            0                   5b4f3e54f955a       pcrf-gtre-1

$ cat crio/crictl_pods | grep -i pcrf-gtre-1
4ce881fa49686       3 days ago          Ready               pcrf-gtre-1                                                       pcrf                                     0                   (default)
5b4f3e54f955a       3 days ago          Ready               pcrf-gtre-1                                                       pcrf                                     0                   (default)

~~~

Also, observing orphan pod errors as well, but those error are getting resolved after kubelet and crio service restart. But now, the duplicate pods are also getting creating again.

~~~
[containers]$ for i in `ls | grep -v logs`; do cat $i | jq 'select (.status.metadata.name == "mme-sgsapre") | "(.status.id) (.status.metadata.name) (.info.sandboxID) (.status.labels."io.kubernetes.pod.name") (.status.state) (.status.startedAt) (.status.createdAt)"'; done
"4a0dca756fb095c5a55af504ad5bc6c5626b275999fabecc1cd996bc06328556 pcrf-gtre 4ce881fa496863d235273fb17d8f42ef8111ef1104f2bffd313be7744b372991 pcrf-gtre-1 CONTAINER_RUNNING 2023-07-28T16:18:12.764587993+03:00 2023-07-28T16:18:12.730942602+03:00"
"d1c96f3bca86e2d8c7110d12eb23985ab70a5930eeb3284bbce156578c3e52a9 pcrf-gtre 5b4f3e54f955a47a0da1834605f258511ec37c556ccadf0c353ba9d28013ef50 pcrf-gtre-1 CONTAINER_RUNNING 2023-07-28T16:00:52.600516696+03:00 2023-07-28T16:00:52.539209941+03:00"

~~~

Those containers are created at 16:00 and 16:18. Logs in comment.

Complete journalctl logs attached as well.

Sos-report attached.