Loading...

XML

Word

Printable

Type: Bug
Resolution: Can't Do
Priority: Minor
Fix Version/s: None
Affects Version/s: 4.12.0
Component/s: Networking / cloud-network-config-controller
Labels:
None

Severity:
Informational
Blocked:
False
Blocked Reason:

Hide

None

Show
None

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

When the admin user associates a floating IP to a neutron reservation port, that port cannot be deleted by the CNCC and logically CloudPrivateIP deletion hangs forever:

$ cat cloudprivateipconfig.yaml 
apiVersion: cloud.network.openshift.io/v1
kind: CloudPrivateIPConfig
metadata:
  annotations:
    k8s.ovn.org/egressip-owner-ref: egressip
  name: 10.196.128.128
spec:
  node: ostest-l8zhs-worker-0-jk29z

(overcloud) [stack@undercloud-0 ~]$ oc apply -f cloudprivateipconfig.yaml 
cloudprivateipconfig.cloud.network.openshift.io/10.196.128.128 created
(overcloud) [stack@undercloud-0 ~]$ openstack floating ip set --port egressip-10.196.128.128 10.0.0.165
(overcloud) [stack@undercloud-0 ~]$ oc delete -f cloudprivateipconfig.yaml 
cloudprivateipconfig.cloud.network.openshift.io "10.196.128.128" deleted
(... hangs here ...)

CNCC logs:

I1215 11:48:37.251248       1 cloudprivateipconfig_controller.go:187] CloudPrivateIPConfig: "10.196.128.128" will be deleted from node: "ostest-l8zhs-worker-0-jk29z"
I1215 11:48:37.271413       1 controller.go:182] Assigning key: 10.196.128.128 to cloud-private-ip-config workqueue
I1215 11:48:38.239482       1 controller.go:182] Assigning key: 10.196.128.128 to cloud-private-ip-config workqueue
E1215 11:48:38.239825       1 controller.go:165] error syncing '10.196.128.128': error releasing CloudPrivateIPConfig: "10.196.128.128" from node: "ostest-l8zhs-worker-0-jk29z", err: Internal Server Error, requeuing in cloud-private-ip-config workqueue
I1215 11:48:38.250342       1 cloudprivateipconfig_controller.go:187] CloudPrivateIPConfig: "10.196.128.128" will be deleted from node: "ostest-l8zhs-worker-0-jk29z"
I1215 11:48:38.268733       1 controller.go:182] Assigning key: 10.196.128.128 to cloud-private-ip-config workqueue
I1215 11:48:39.222875       1 controller.go:182] Assigning key: 10.196.128.128 to cloud-private-ip-config workqueue
E1215 11:48:39.232588       1 controller.go:165] error syncing '10.196.128.128': error releasing CloudPrivateIPConfig: "10.196.128.128" from node: "ostest-l8zhs-worker-0-jk29z", err: Internal Server Error, requeuing in cloud-private-ip-config workqueue
I1215 11:48:39.238750       1 cloudprivateipconfig_controller.go:187] CloudPrivateIPConfig: "10.196.128.128" will be deleted from node: "ostest-l8zhs-worker-0-jk29z"
I1215 11:48:39.250368       1 controller.go:182] Assigning key: 10.196.128.128 to cloud-private-ip-config workqueue
I1215 11:48:40.020576       1 controller.go:182] Assigning key: 10.196.128.128 to cloud-private-ip-config workqueue
E1215 11:48:40.023658       1 controller.go:165] error syncing '10.196.128.128': error releasing CloudPrivateIPConfig: "10.196.128.128" from node: "ostest-l8zhs-worker-0-jk29z", err: Internal Server Error, requeuing in cloud-private-ip-config workqueue
I1215 11:48:40.050849       1 cloudprivateipconfig_controller.go:187] CloudPrivateIPConfig: "10.196.128.128" will be deleted from node: "ostest-l8zhs-worker-0-jk29z"
I1215 11:48:40.074774       1 controller.go:182] Assigning key: 10.196.128.128 to cloud-private-ip-config workqueue
I1215 11:48:40.783631       1 controller.go:182] Assigning key: 10.196.128.128 to cloud-private-ip-config workqueue
E1215 11:48:40.785174       1 controller.go:165] error syncing '10.196.128.128': error releasing CloudPrivateIPConfig: "10.196.128.128" from node: "ostest-l8zhs-worker-0-jk29z", err: Internal Server Error, requeuing in cloud-private-ip-config workqueue

neutron logs (on one of the controller nodes in /var/log/containers/neutron/server.log:

2022-12-15 11:48:34.083 21 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 724, in _read_query_result
2022-12-15 11:48:34.083 21 ERROR neutron.pecan_wsgi.hooks.translation     result.read()
2022-12-15 11:48:34.083 21 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 1069, in read
2022-12-15 11:48:34.083 21 ERROR neutron.pecan_wsgi.hooks.translation     first_packet = self.connection._read_packet()
2022-12-15 11:48:34.083 21 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 676, in _read_packet
2022-12-15 11:48:34.083 21 ERROR neutron.pecan_wsgi.hooks.translation     packet.raise_for_error()
2022-12-15 11:48:34.083 21 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/protocol.py", line 223, in raise_for_error
2022-12-15 11:48:34.083 21 ERROR neutron.pecan_wsgi.hooks.translation     err.raise_mysql_exception(self._data)
2022-12-15 11:48:34.083 21 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/err.py", line 107, in raise_mysql_exception
2022-12-15 11:48:34.083 21 ERROR neutron.pecan_wsgi.hooks.translation     raise errorclass(errno, errval)
2022-12-15 11:48:34.083 21 ERROR neutron.pecan_wsgi.hooks.translation oslo_db.exception.DBReferenceError: (pymysql.err.IntegrityError) (1451, 'Cannot delete or update a parent row: a foreign key constraint fails (`ovs_neutron`.`floatingips`, CONSTRAINT `floatingips_ibfk_1` FOREIGN KEY (`fixed_port_id`) REFERENCES `ports` (`id`))')
2022-12-15 11:48:34.083 21 ERROR neutron.pecan_wsgi.hooks.translation [SQL: DELETE FROM ports WHERE ports.id = %(id)s]
2022-12-15 11:48:34.083 21 ERROR neutron.pecan_wsgi.hooks.translation [parameters: {'id': 'c0929f45-431d-4c82-8988-01b057b0009d'}]
2022-12-15 11:48:34.083 21 ERROR neutron.pecan_wsgi.hooks.translation (Background on this error at: http://sqlalche.me/e/gkpj)
2022-12-15 11:48:34.083 21 ERROR neutron.pecan_wsgi.hooks.translation 
2022-12-15 11:48:34.085 21 DEBUG neutron.pecan_wsgi.hooks.notifier [req-6faaa59d-77ea-45d5-9e9e-ca68a5e9c10b 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] No notification will be sent due to unsuccessful status code: 500 after /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/notifier.py:79
2022-12-15 11:48:34.087 21 INFO neutron.wsgi [req-6faaa59d-77ea-45d5-9e9e-ca68a5e9c10b 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] 172.17.1.39 "DELETE /v2.0/ports/c0929f45-431d-4c82-8988-01b057b0009d HTTP/1.1" status: 500  len: 344 time: 0.5001817
2022-12-15 11:48:34.147 20 DEBUG neutron.wsgi [-] (20) accepted ('172.17.1.111', 41292) server /usr/lib/python3.6/site-packages/eventlet/wsgi.py:985
2022-12-15 11:48:34.213 20 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-625f07ed-ac68-42bf-812a-c84740141420 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:34.218 20 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-625f07ed-ac68-42bf-812a-c84740141420 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:34.219 20 INFO neutron.wsgi [req-625f07ed-ac68-42bf-812a-c84740141420 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] 172.17.1.111 "GET /v2.0/ports?device_id=5ba35d15-916a-4229-986a-e78ecf06328e HTTP/1.1" status: 200  len: 2822 time: 0.0711522
2022-12-15 11:48:34.265 18 DEBUG neutron.wsgi [-] (18) accepted ('172.17.1.39', 34504) server /usr/lib/python3.6/site-packages/eventlet/wsgi.py:985
2022-12-15 11:48:34.340 18 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29a11f5d-1fbf-4427-af2d-049ef4f45790 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:34.344 18 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29a11f5d-1fbf-4427-af2d-049ef4f45790 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:34.348 18 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29a11f5d-1fbf-4427-af2d-049ef4f45790 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:34.352 18 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29a11f5d-1fbf-4427-af2d-049ef4f45790 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:34.356 18 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29a11f5d-1fbf-4427-af2d-049ef4f45790 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:34.360 18 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29a11f5d-1fbf-4427-af2d-049ef4f45790 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:34.363 18 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29a11f5d-1fbf-4427-af2d-049ef4f45790 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:34.367 18 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29a11f5d-1fbf-4427-af2d-049ef4f45790 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:34.372 18 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29a11f5d-1fbf-4427-af2d-049ef4f45790 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:34.376 18 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29a11f5d-1fbf-4427-af2d-049ef4f45790 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:34.380 18 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29a11f5d-1fbf-4427-af2d-049ef4f45790 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:34.381 18 INFO neutron.wsgi [req-29a11f5d-1fbf-4427-af2d-049ef4f45790 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] 172.17.1.39 "GET /v2.0/ports?network_id=505ba282-246d-49b8-b574-0e9dda638b7d HTTP/1.1" status: 200  len: 12742 time: 0.1159384
2022-12-15 11:48:34.453 15 DEBUG neutron.wsgi [-] (15) accepted ('172.17.1.39', 34520) server /usr/lib/python3.6/site-packages/eventlet/wsgi.py:985
2022-12-15 11:48:34.457 15 INFO neutron.wsgi [-] 172.17.1.39 "OPTIONS / HTTP/1.0" status: 200  len: 249 time: 0.0034254
2022-12-15 11:48:34.465 15 DEBUG neutron.wsgi [-] (15) accepted ('172.17.1.49', 37088) server /usr/lib/python3.6/site-packages/eventlet/wsgi.py:985
2022-12-15 11:48:34.469 15 INFO neutron.wsgi [-] 172.17.1.49 "OPTIONS / HTTP/1.0" status: 200  len: 249 time: 0.0032351
2022-12-15 11:48:34.989 20 DEBUG neutron.wsgi [-] (20) accepted ('172.17.1.39', 34556) server /usr/lib/python3.6/site-packages/eventlet/wsgi.py:985
2022-12-15 11:48:35.039 20 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9ed47080-89e6-4063-a740-3fb704142e15 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['shared'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:35.040 20 INFO neutron.wsgi [req-9ed47080-89e6-4063-a740-3fb704142e15 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] 172.17.1.39 "GET /v2.0/subnets?network_id=505ba282-246d-49b8-b574-0e9dda638b7d HTTP/1.1" status: 200  len: 877 time: 0.0508268
2022-12-15 11:48:35.702 15 DEBUG neutron.wsgi [-] (15) accepted ('172.17.1.39', 34596) server /usr/lib/python3.6/site-packages/eventlet/wsgi.py:985
2022-12-15 11:48:35.796 15 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-595819d9-9a52-4e60-9450-80e5aa96e515 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:35.804 15 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-595819d9-9a52-4e60-9450-80e5aa96e515 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:35.807 15 INFO neutron.wsgi [req-595819d9-9a52-4e60-9450-80e5aa96e515 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] 172.17.1.39 "GET /v2.0/ports?device_id=f6b98b87-641b-4e1a-9a75-00a3d04e88a4&device_owner=compute%3Anova HTTP/1.1" status: 200  len: 2825 time: 0.1035790
2022-12-15 11:48:35.979 22 DEBUG neutron.wsgi [-] (22) accepted ('172.17.1.39', 34608) server /usr/lib/python3.6/site-packages/eventlet/wsgi.py:985
2022-12-15 11:48:36.030 22 DEBUG neutron.plugins.ml2.plugin [req-f48fe0f3-969d-4137-b1a7-841fd654dfdc 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Deleting port c0929f45-431d-4c82-8988-01b057b0009d _pre_delete_port /usr/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py:1924
2022-12-15 11:48:36.030 22 DEBUG neutron_lib.callbacks.manager [req-f48fe0f3-969d-4137-b1a7-841fd654dfdc 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Notify callbacks ['neutron.plugins.ml2.extension
s.dns_integration._delete_port_in_external_dns_service-8775481335257', 'neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8775481426223', 'neutron.services.trunk.rules.enforce_port_deletion_rules-8775474732966'] for port, before_delete _notify_loop /usr/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py:193
2022-12-15 11:48:36.037 21 DEBUG neutron.wsgi [-] (21) accepted ('172.17.1.111', 41392) server /usr/lib/python3.6/site-packages/eventlet/wsgi.py:985
2022-12-15 11:48:36.041 21 INFO neutron.wsgi [-] 172.17.1.111 "OPTIONS / HTTP/1.0" status: 200  len: 249 time: 0.0035007
2022-12-15 11:48:36.227 22 DEBUG neutron_lib.callbacks.manager [req-f48fe0f3-969d-4137-b1a7-841fd654dfdc 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Notify callbacks ['networking_ovn.l3.l3_ovn.L3_NAT_dbonly_mixin._precommit_delete_port_callback-8087996'] for port, precommit_delete _notify_loop /usr/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py:193
2022-12-15 11:48:36.459 20 DEBUG neutron.wsgi [-] (20) accepted ('172.17.1.39', 34634) server /usr/lib/python3.6/site-packages/eventlet/wsgi.py:985
2022-12-15 11:48:36.463 20 INFO neutron.wsgi [-] 172.17.1.39 "OPTIONS / HTTP/1.0" status: 200  len: 249 time: 0.0030916
2022-12-15 11:48:36.471 20 DEBUG neutron.wsgi [-] (20) accepted ('172.17.1.49', 37188) server /usr/lib/python3.6/site-packages/eventlet/wsgi.py:985
2022-12-15 11:48:36.475 20 INFO neutron.wsgi [-] 172.17.1.49 "OPTIONS / HTTP/1.0" status: 200  len: 249 time: 0.0032487
2022-12-15 11:48:36.715 22 DEBUG networking_ovn.common.ovn_client [req-f48fe0f3-969d-4137-b1a7-841fd654dfdc 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] FIP {'id': '3611d344-3196-4d99-917e-593174c22d00', 'logical_ip': '10.196.128.128', 'external_ip': '10.0.0.165', 'floating_network_id': '3539b208-8038-4c02-8e58-a09b9bcdf0f7'} doesn't have external_ids. _delete_floatingip /usr/lib/python3.6/site-packages/networking_ovn/common/ovn_client.py:919
2022-12-15 11:48:36.720 22 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DeleteNATRuleInLRouterCommand(lrouter=neutron-cf9ba24b-748e-470d-acbb-4200d69b8cc7, type=dnat_and_snat, logical_ip=10.196.128.128, external_ip=10.0.0.165, if_exists=True) do_commit /usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89
2022-12-15 11:48:36.721 22 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): QoSDelExtIdCommand(lswitch=neutron-3539b208-8038-4c02-8e58-a09b9bcdf0f7, external_ids={'neutron:fip_id': '3611d344-3196-4d99-917e-593174c22d00'}, if_exists=True) do_commit /usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89
2022-12-15 11:48:36.721 22 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change do_commit /usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:139
2022-12-15 11:48:36.768 22 ERROR networking_ovn.l3.l3_ovn [req-f48fe0f3-969d-4137-b1a7-841fd654dfdc 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Error in disassociating floatingip 3611d344-3196-4d99-917e-593174c22d00: 'NoneType' object is not subscriptable
2022-12-15 11:48:36.769 22 DEBUG neutron.plugins.ml2.plugin [req-f48fe0f3-969d-4137-b1a7-841fd654dfdc 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Calling delete_port for c0929f45-431d-4c82-8988-01b057b0009d owned by OpenShiftEgressIP delete_port /usr/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py:1997
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation [req-f48fe0f3-969d-4137-b1a7-841fd654dfdc 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] DELETE failed.: oslo_db.exception.DBReferenceError: (pymysql.err.IntegrityError) (1451, 'Cannot delete or update a parent row: a foreign key constraint fails (`ovs_neutron`.`floatingips`, CONSTRAINT `floatingips_ibfk_1` FOREIGN KEY (`fixed_port_id`) REFERENCES `ports` (`id`))')
[SQL: DELETE FROM ports WHERE ports.id = %(id)s]
[parameters: {'id': 'c0929f45-431d-4c82-8988-01b057b0009d'}]
(Background on this error at: http://sqlalche.me/e/gkpj)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation Traceback (most recent call last):
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     cursor, statement, parameters, context
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     cursor.execute(statement, parameters)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/cursors.py", line 163, in execute
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     result = self._query(query)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/cursors.py", line 321, in _query
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     conn.query(q)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 505, in query
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 724, in _read_query_result
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     result.read()
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 1069, in read
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     first_packet = self.connection._read_packet()
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 676, in _read_packet
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     packet.raise_for_error()
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/protocol.py", line 223, in raise_for_error
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     err.raise_mysql_exception(self._data)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/err.py", line 107, in raise_mysql_exception
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     raise errorclass(errno, errval)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation pymysql.err.IntegrityError: (1451, 'Cannot delete or update a parent row: a foreign key constraint fails (`ovs_neutron`.`floatingips`, CONSTRAINT `floatingips_ibfk_1` FOREIGN KEY (`fixed_port_id`) REFERENCES `ports` (`id`))')
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation 
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation The above exception was the direct cause of the following exception:
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation 
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation Traceback (most recent call last):
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pecan/core.py", line 683, in __call__
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     self.invoke_controller(controller, args, kwargs, state)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pecan/core.py", line 574, in invoke_controller
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     result = controller(*args, **kwargs)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     setattr(e, '_RETRY_EXCEEDED', True)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     raise value
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     return f(*args, **kwargs)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     ectxt.value = e.inner_exc
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     raise value
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     return f(*args, **kwargs)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     LOG.debug("Retry wrapper got retriable exception: %s", e)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     raise value
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     return f(*dup_args, **dup_kwargs)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron/pecan_wsgi/controllers/utils.py", line 76, in wrapped
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     return f(*args, **kwargs)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron/pecan_wsgi/controllers/utils.py", line 115, in wrapped
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     f(*args, **kwargs)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron/pecan_wsgi/controllers/resource.py", line 85, in delete
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     return self.plugin_deleter(*deleter_args)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 699, in inner
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     return f(*args, **kwargs)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 233, in wrapped
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     return method(*args, **kwargs)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     setattr(e, '_RETRY_EXCEEDED', True)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     raise value
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     return f(*args, **kwargs)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     ectxt.value = e.inner_exc
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     raise value
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     return f(*args, **kwargs)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     LOG.debug("Retry wrapper got retriable exception: %s", e)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     raise value
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     return f(*dup_args, **dup_kwargs)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1998, in delete_port
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     super(Ml2Plugin, self).delete_port(context, id)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 233, in wrapped
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     return method(*args, **kwargs)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron/db/db_base_plugin_v2.py", line 1520, in delete_port
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     self.ipam.delete_port(context, id)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron/db/ipam_pluggable_backend.py", line 519, in delete_port
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     port['fixed_ips'])
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron/db/ipam_pluggable_backend.py", line 110, in _ipam_deallocate_ips
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     "external system for %s", addresses)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     self.force_reraise()
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     six.reraise(self.type_, self.value, self.tb)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     raise value
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron/db/ipam_pluggable_backend.py", line 86, in _ipam_deallocate_ips
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     ipam_subnet = ipam_driver.get_subnet(ip['subnet_id'])
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron/ipam/drivers/neutrondb_ipam/driver.py", line 331, in get_subnet
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     return NeutronDbSubnet.load(subnet_id, self._context)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron/ipam/drivers/neutrondb_ipam/driver.py", line 94, in load
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     ctx, neutron_subnet_id)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron/ipam/drivers/neutrondb_ipam/db_api.py", line 30, in load_by_neutron_subnet_id
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     context, neutron_subnet_id=neutron_subnet_id)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron/objects/base.py", line 640, in get_objects
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     cls, context, _pager=_pager, **cls.modify_fields_to_db(kwargs))
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron/objects/db/api.py", line 52, in get_objects
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     **(_pager.to_kwargs(context, obj_cls) if _pager else {}))
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/neutron_lib/db/model_query.py", line 311, in get_collection
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     for c in query
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 3316, in __iter__
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     self.session._autoflush()
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 1576, in _autoflush
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     self.flush()
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 2451, in flush
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     self._flush(objects)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 2589, in _flush
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     transaction.rollback(_capture_exception=True)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     compat.reraise(exc_type, exc_value, exc_tb)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 129, in reraise
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     raise value
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 2549, in _flush
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     flush_context.execute()
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     rec.execute(self)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 624, in execute
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     uow,
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 348, in delete_obj
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     delete,
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 1305, in _emit_delete_statements
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     c = connection.execute(statement, del_objects)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     return meth(self, multiparams, params)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     return connection._execute_clauseelement(self, multiparams, params)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1107, in _execute_clauseelement
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     distilled_params,
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     e, statement, parameters, cursor, context
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1464, in _handle_dbapi_exception
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     util.raise_from_cause(newraise, exc_info)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     reraise(type(exception), exception, tb=exc_tb, cause=cause)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     raise value.with_traceback(tb)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     cursor, statement, parameters, context
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     cursor.execute(statement, parameters)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/cursors.py", line 163, in execute
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     result = self._query(query)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/cursors.py", line 321, in _query
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     conn.query(q)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 505, in query
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 724, in _read_query_result
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     result.read()
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 1069, in read
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     first_packet = self.connection._read_packet()
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 676, in _read_packet
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     packet.raise_for_error()
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/protocol.py", line 223, in raise_for_error
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     err.raise_mysql_exception(self._data)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation   File "/usr/lib/python3.6/site-packages/pymysql/err.py", line 107, in raise_mysql_exception
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation     raise errorclass(errno, errval)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation oslo_db.exception.DBReferenceError: (pymysql.err.IntegrityError) (1451, 'Cannot delete or update a parent row: a foreign key constraint fails (`ovs_neutron`.`floatingips`, CONSTRAINT `floatingips_ibfk_1` FOREIGN KEY (`fixed_port_id`) REFERENCES `ports` (`id`))')
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation [SQL: DELETE FROM ports WHERE ports.id = %(id)s]
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation [parameters: {'id': 'c0929f45-431d-4c82-8988-01b057b0009d'}]
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation (Background on this error at: http://sqlalche.me/e/gkpj)
2022-12-15 11:48:36.827 22 ERROR neutron.pecan_wsgi.hooks.translation 
2022-12-15 11:48:36.829 22 DEBUG neutron.pecan_wsgi.hooks.notifier [req-f48fe0f3-969d-4137-b1a7-841fd654dfdc 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] No notification will be sent due to unsuccessful status code: 500 after /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/notifier.py:79
2022-12-15 11:48:36.831 22 INFO neutron.wsgi [req-f48fe0f3-969d-4137-b1a7-841fd654dfdc 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] 172.17.1.39 "DELETE /v2.0/ports/c0929f45-431d-4c82-8988-01b057b0009d HTTP/1.1" status: 500  len: 344 time: 0.8517702
2022-12-15 11:48:37.022 20 DEBUG neutron.wsgi [-] (20) accepted ('172.17.1.39', 34676) server /usr/lib/python3.6/site-packages/eventlet/wsgi.py:985
2022-12-15 11:48:37.092 20 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-890595f6-f02f-4fed-9dce-102c5d3f0f09 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256
2022-12-15 11:48:37.096 20 DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-890595f6-f02f-4fed-9dce-102c5d3f0f09 6633cbd343324af3a4e11f64f28d19c6 3d53d2a8c47a440cb1a9b3f21a9d9ca0 - default default] Attributes excluded by policy engine: ['binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'resource_request'] _exclude_attributes_by_policy /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/policy_enforcement.py:256

Solution:

openstack floating ip unset --port 10.0.0.165

Assignee:: Andreas Karis

Reporter:: Andreas Karis

QA Contact:: Anurag Saxena

Contributing Groups:: Red Hat Employee

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Created:: 2022/12/15 11:54 AM

Updated:: 2023/01/01 4:34 PM

Resolved:: 2023/01/01 4:34 PM

Details

Description

Attachments

Activity

People

Dates