For the "src" attribute (RTA_SRC) of a route, kernel requires that such an address is actually configured. It doesn't need to be on the same interface though...

When the address gets removed, kernel will also remove the route with the corresponding "src" attribute. However, it fails to send RTM_DELROUTE notification:

>>>>
#!/bin/bash

set -ex

ip netns del x &>/dev/null || :
ip netns add x

ip -netns x link add net1 type dummy
ip -netns x link add net2 type dummy
ip -netns x link set net1 up
ip -netns x link set net2 up

ip -netns x addr add 192.168.5.5/24 dev net1
ip -netns x route append 7.7.7.0/24 dev net2 src 192.168.5.5

ip -netns x -4 addr
ip -netns x -4 route

ip -netns x monitor addr &
ip -netns x monitor route &

sleep 0.2

ip -netns x addr del 192.168.5.5/24 dev net1
ip -netns x -4 route

sleep 0.2

kill – $(jobs -p)
wait
wait
<<<<

On rhel-9.2 (5.14.0-267.el9.x86_64) and Fedora 37 (6.1.8-200.fc37.x86_64) above script gives:

<<<<
+ ip netns del x
+ ip netns add x
+ ip -netns x link add net1 type dummy
+ ip -netns x link add net2 type dummy
+ ip -netns x link set net1 up
+ ip -netns x link set net2 up
+ ip -netns x addr add 192.168.5.5/24 dev net1
+ ip -netns x route append 7.7.7.0/24 dev net2 src 192.168.5.5
+ ip -netns x -4 addr
2: net1: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
inet 192.168.5.5/24 scope global net1
valid_lft forever preferred_lft forever
+ ip -netns x -4 route
7.7.7.0/24 dev net2 scope link src 192.168.5.5
192.168.5.0/24 dev net1 proto kernel scope link src 192.168.5.5
+ ip -netns x monitor addr
+ sleep 0.2
+ ip -netns x monitor route
+ ip -netns x addr del 192.168.5.5/24 dev net1
Deleted 2: net1 inet 192.168.5.5/24 scope global net1
Deleted 192.168.5.0/24 dev net1 proto kernel scope link src 192.168.5.5
valid_lft forever preferred_lft forever
Deleted broadcast 192.168.5.255 dev net1 table local proto kernel scope link src 192.168.5.5
Deleted local 192.168.5.5 dev net1 table local proto kernel scope host src 192.168.5.5
+ ip -netns x -4 route
+ sleep 0.2
++ jobs -p
+ kill – 18111 18112
+ wait
+ wait
>>>

Note that there is no notification about the route on net2 being removed, but it's not longer there.

Sending accurate events is important to NetworkManager, because it keeps a cache of the routes. Missing/Wrong events mean that the cache becomes inconsistent.