Description of problem:
When run ptp4l using udpv4 on a port without ipv4 address, it won't report error. In this case the address in SYNC packets is from another port.

1. ip addr show ens4f0np0
   6: ens4f0np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
   link/ether 00:0f:53:7f:88:a0 brd ff:ff:ff:ff:ff:ff
   altname enp175s0f0np0
   inet6 fe80::20f:53ff:fe7f:88a0/64 scope link
   valid_lft forever preferred_lft forever
2. ptp4l -EH4mi ens4f0np0
   ptp4l[72743.712]: selected /dev/ptp4 as PTP clock
   ptp4l[72743.713]: port 1: INITIALIZING to LISTENING on INIT_COMPLETE
   ptp4l[72743.713]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE
   ptp4l[72743.713]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE
   ptp4l[72751.711]: port 1: LISTENING to MASTER on ANNOUNCE_RECEIPT_TIMEOUT_EXPIRES
   ptp4l[72751.711]: selected local clock 000f53.fffe.7f88a0 as best master
   ptp4l[72751.711]: port 1: assuming the grand master role
   Version-Release number of selected component (if applicable):

The address in SYNC packets is assigned to another port in this server:

1. ip addr show eno1
   2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
   link/ether 34:73:5a:99:97:ac brd ff:ff:ff:ff:ff:ff
   altname enp24s0f0
   inet 10.73.131.75/23 brd 10.73.131.255 scope global dynamic noprefixroute eno1
   valid_lft 35939sec preferred_lft 35939sec
   inet6 2620:52:0:4982:3673:5aff:fe99:97ac/64 scope global dynamic noprefixroute
   valid_lft 2591998sec preferred_lft 604798sec
   inet6 fe80::3673:5aff:fe99:97ac/64 scope link noprefixroute
   valid_lft forever preferred_lft forever

Frame 3: 86 bytes on wire (688 bits), 86 bytes captured (688 bits) on interface 0
Interface id: 0 (ens4f0np0)
Interface name: ens4f0np0
Encapsulation type: Ethernet (1)
Arrival Time: Nov 28, 2022 20:27:54.447241961 EST
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1669685274.447241961 seconds
[Time delta from previous captured frame: 0.058639484 seconds]
[Time delta from previous displayed frame: 0.058639484 seconds]
[Time since reference or first frame: 0.059437985 seconds]
Frame Number: 3
Frame Length: 86 bytes (688 bits)
Capture Length: 86 bytes (688 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:udp:ptp]
Ethernet II, Src: Solarfla_7f:88:a0 (00:0f:53:7f:88:a0), Dst: IPv4mcast_01:81 (01:00:5e:00:01:81)
Destination: IPv4mcast_01:81 (01:00:5e:00:01:81)
Address: IPv4mcast_01:81 (01:00:5e:00:01:81)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
Source: Solarfla_7f:88:a0 (00:0f:53:7f:88:a0)
Address: Solarfla_7f:88:a0 (00:0f:53:7f:88:a0)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.73.131.75, Dst: 224.0.1.129
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
0000 00.. = Differentiated Services Codepoint: Default (0)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 72
Identification: 0x72aa (29354)
Flags: 0x4000, Don't fragment
0... .... .... .... = Reserved bit: Not set
.1.. .... .... .... = Don't fragment: Set
..0. .... .... .... = More fragments: Not set
...0 0000 0000 0000 = Fragment offset: 0
Time to live: 1
Protocol: UDP (17)
Header checksum: 0x97e5 [validation disabled]
[Header checksum status: Unverified]
Source: 10.73.131.75
Destination: 224.0.1.129
User Datagram Protocol, Src Port: 319, Dst Port: 319
Source Port: 319
Destination Port: 319
Length: 52
Checksum: 0x6f5b [unverified]
[Checksum Status: Unverified]
[Stream index: 2]
Precision Time Protocol (IEEE1588)
0000 .... = transportSpecific: 0x0
...0 .... = V1 Compatibility: False
.... 0000 = messageId: Sync Message (0x0)
.... 0010 = versionPTP: 2
messageLength: 44
subdomainNumber: 0
flags: 0x0200
0... .... .... .... = PTP_SECURITY: False
.0.. .... .... .... = PTP profile Specific 2: False
..0. .... .... .... = PTP profile Specific 1: False
.... .0.. .... .... = PTP_UNICAST: False
.... ..1. .... .... = PTP_TWO_STEP: True
.... ...0 .... .... = PTP_ALTERNATE_MASTER: False
.... .... ..0. .... = FREQUENCY_TRACEABLE: False
.... .... ...0 .... = TIME_TRACEABLE: False
.... .... .... 0... = PTP_TIMESCALE: False
.... .... .... .0.. = PTP_UTC_REASONABLE: False
.... .... .... ..0. = PTP_LI_59: False
.... .... .... ...0 = PTP_LI_61: False
correction: 0.000000 nanoseconds
correction: Ns: 0 nanoseconds
correctionSubNs: 0 nanoseconds
ClockIdentity: 0x000f53fffe7f88a0
SourcePortID: 1
sequenceId: 66
control: Sync Message (0)
logMessagePeriod: 0
originTimestamp (seconds): 0
originTimestamp (nanoseconds): 0

How reproducible:
always

Steps to Reproduce:
1.run ptp4l on a port without ipv4 address assigned
2.
3.

Actual results:
ptp4l doesn't report error and using a fault v4 address

Expected results:
ptp4l should report a error

Additional info: