Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-24803

Ovsdb-client restore fails with "Transaction Failed(End of file)" Error during RHOSP to RHOSO OVN DB Migration

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • None
    • None
    • data-plane-adoption
    • None
    • Neutron Quark 5
    • 1
    • Critical

      Description of the problem:
      During RHOSP 17.1 to RHOSO 18.0 OVN migration, ovsdb-client restore fails with "transaction failed (End of file)" error when restoring OVN NB/SB databases.

      Logs:
      [root@xxx tmp]# cat ovn-sb-restore.log | tail -n 20
      2026-01-05T07:42:11Z|13572|stream_ssl|DBG|client0-->ssl:172.30.7.133:6642 type 257 (1 bytes)
      2026-01-05T07:42:11Z|13573|stream_ssl|DBG|client0-->ssl:172.30.7.133:6642 type 256 (5 bytes)
      2026-01-05T07:42:11Z|13574|stream_ssl|DBG|client0-->ssl:172.30.7.133:6642 type 257 (1 bytes)
      2026-01-05T07:42:11Z|13575|stream_ssl|DBG|client0-->ssl:172.30.7.133:6642 type 256 (5 bytes)
      2026-01-05T07:42:11Z|13576|stream_ssl|DBG|client0-->ssl:172.30.7.133:6642 type 257 (1 bytes)
      2026-01-05T07:42:17Z|13577|poll_loop|DBG|wakeup due to [POLLIN] on fd 3 (10.128.0.240:55962<->172.30.7.133:6642) at ../lib/stream-ssl.c:835 (1% CPU usage)
      2026-01-05T07:42:17Z|13578|stream_ssl|DBG|client0<--ssl:172.30.7.133:6642 type 256 (5 bytes)
      2026-01-05T07:42:17Z|13579|poll_loop|DBG|wakeup due to [POLLIN] on fd 3 (10.128.0.240:55962<->172.30.7.133:6642) at ../lib/stream-ssl.c:835 (1% CPU usage)
      2026-01-05T07:42:17Z|13580|stream_ssl|DBG|client0<--ssl:172.30.7.133:6642 type 257 (1 bytes)
      2026-01-05T07:42:17Z|13581|stream_ssl|DBG|client0<--ssl:172.30.7.133:6642 type 256 (5 bytes)
      2026-01-05T07:42:17Z|13582|stream_ssl|DBG|client0<--ssl:172.30.7.133:6642 type 257 (1 bytes)
      2026-01-05T07:42:17Z|13583|stream_ssl|DBG|client0<--ssl:172.30.7.133:6642 type 256 (5 bytes)
      2026-01-05T07:42:17Z|13584|stream_ssl|DBG|client0<--ssl:172.30.7.133:6642 type 257 (1 bytes)
      2026-01-05T07:42:17Z|13585|stream_ssl|DBG|client0<--ssl:172.30.7.133:6642 type 256 (5 bytes)
      2026-01-05T07:42:18Z|13586|poll_loop|DBG|wakeup due to [POLLIN] on fd 3 (10.128.0.240:55962<->172.30.7.133:6642) at ../lib/stream-ssl.c:835 (1% CPU usage)
      2026-01-05T07:42:18Z|13587|stream_ssl|DBG|client0-->ssl:172.30.7.133:6642 type 256 (5 bytes)
      2026-01-05T07:42:18Z|13588|stream_ssl|DBG|client0-->ssl:172.30.7.133:6642 type 257 (1 bytes)
      2026-01-05T07:42:18Z|13589|stream_ssl|DBG|client0-->ssl:172.30.7.133:6642 alert: warning, close_notify (2 bytes)
      ovsdb-client: transaction failed (End of file)
      command terminated with exit code 1

      Root Cause:
      Connection table configuration mismatch between source and target:
      RHOSP 17.1 backup: pssl:6641:0.0.0.0 (IPv4)
      RHOSO 18.0 fresh: pssl:6641:[::] (IPv6)

      [root@xxxx chapter_3]# oc exec ovn-copy-data -n openstack – ovsdb-tool query /backup/ovs-sb.db   '["OVN_Southbound", {"op": "select", "table": "Connection", "where": []}]' | jq '.'
      [
        {
          "rows": [
           

      {         "_uuid": [           "uuid",           "c01ea1af-df34-43b3-b01d-ce7ec04ec09d"         ],         "_version": [           "uuid",           "8583525a-3994-47fc-91c3-96942cae1fd8"         ],         "external_ids": [           "map",           []         ],         "inactivity_probe": 60000,         "is_connected": false,         "max_backoff": [           "set",           []         ],         "other_config": [           "map",           []         ],         "read_only": false,         "role": "",         "status": [           "map",           []         ],         "target": "pssl:6642:0.0.0.0"       }

          ]
        }
      ]

       

      [root@xxx chapter_3]# oc exec -n openstack ovsdbserver-sb-0 – ovn-sbctl list connection
      Defaulted container "ovsdbserver-sb" out of: ovsdbserver-sb, openstack-network-exporter
      _uuid               : 8f489214-15b4-4df1-9998-911c90a6ac41
      external_ids        : {}
      inactivity_probe    : 60000
      is_connected        : false
      max_backoff         : []
      other_config        : {}
      read_only           : false
      role                : ""
      status              : {}
      target              : "pssl:6642:[::]"

      So, when restore transaction includes Connection table change, server immediately reconfigures listening sockets, closing the client connection before sending the response.

      So the workaround is to set target connection table to match the backup.

              twilson@redhat.com Terry Wilson
              rpulapak@redhat.com Rajesh Pulapakula
              rhos-dfg-networking-squad-neutron
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: