Description of problem:
After changed the kata container's resolv.conf and using ping, "crictl rm" will hang.

[EDIT - 2023/05/22 - Julien Ropé]

crictl rm hangs in some situations. 

From the comments section, there seem to be multiple leads:

• the issue happens when deleting the container before it is started? Latest comment don't mention that anymore, to be verified
• the issue happens because the kata agent doesn't find the process it is supposed to kill, and the error handling prevents going further. The reason why it doesn't find it is unclear.

Also important to note:

• changing resolv.conf is a red herring (see first comment below)
• even "exec" seems to not be needed to reproduce the problem

I'm keeping the rest of the description below for reference, but most of the information seems to be in the comments now. 

[/EDIT]

1. crictl rm 3a35a3ad68940
   ERRO[0130] removing container "3a35a3ad68940" failed: rpc error: code = Unknown desc = unable to stop container 3a35a3ad68940484a255a808d45a2d4113e43d36761be7478d293844fd14df6d: failed to stop container 3a35a3ad68940484a255a808d45a2d4113e43d36761be7478d293844fd14df6d: StopContainer with signal killed timed out after (2m0s)
   FATA[0130] unable to remove container(s)

Steps:

1. cat pod-config.json { "metadata": { "name": "alpine-sandbox", "namespace": "default", "attempt": 1, "uid": "hdishd83djaidwnduwk28bcsb" },
   "log_directory": "/tmp",
   "linux": {
   }
   }

1. cat container-pod.json { "metadata": { "name": "alpine" },
   "image": { "image": "alpine" }

   ,
   "command": [
   "sleep", "3600"
   ],
   "log_path":"alpine.0.log",
   "linux":
   { "security_context": Unknown macro:
   

   { "capabilities"}

   }
   }

1. crictl runp --runtime=kata pod-config.json
   d1993b6a0479657f928b3b7f9d427ec2e8cea1a1898f2e1be993081e7eb43130

1. crictl create d1993b6a0479657f928b3b7f9d427ec2e8cea1a1898f2e1be993081e7eb43130 container-pod.json pod-config.json
   30e947cd0a85eab9634cef2975acd18d3e98367eeac1826506cea9d6d0bb784b

1. crictl exec -it 30e947cd0a85eab9634cef2975acd18d3e98367eeac1826506cea9d6d0bb784b sh

1. Inside the container, changed the resolv.conf to use:
   nameserver 10.19.42.41
   nameserver 10.11.5.19
   nameserver 10.5.30.160

1. ping www.google.com
   PING www.google.com (74.125.68.105): 56 data bytes
   64 bytes from 74.125.68.105: seq=0 ttl=100 time=255.879 ms
   64 bytes from 74.125.68.105: seq=1 ttl=100 time=256.039 ms
   ...

1. exit

1. crictl rm 3a35a3ad68940

Version-Release number of selected component (if applicable):
kata-agent-1.11.3-1.el8.x86_64
kata-shim-1.11.3-1.el8.x86_64
kata-osbuilder-1.11.3-1.el8.x86_64
kata-runtime-1.11.3-1.el8.x86_64
cri-o-1.20.0-0.rhaos4.7.git845747f.el8.40.x86_64
cri-tools-1.20.0-1.el8.x86_64

How reproducible:
always