Uploaded image for project: 'OCP Technical Release Team'
  1. OCP Technical Release Team
  2. TRT-848

azure ci jobs are permafailing

    XMLWordPrintable

Details

    • Story
    • Resolution: Done
    • Blocker
    • None
    • None
    • False
    • None
    • False

    Description

      Azure CI jobs are permafailing with the sdn drop-icmp container crashlooping with errors like this:

       

       

      2023-02-13T11:33:43.533651259Z + touch /var/run/add_iptables.sh
      2023-02-13T11:33:43.536071373Z + chmod 0755 /var/run/add_iptables.sh
      2023-02-13T11:33:43.538207086Z + cat
      2023-02-13T11:33:43.541084503Z ++ date '+%m%d %H:%M:%S.%N'
      2023-02-13T11:33:43.543265915Z + echo 'I0213 11:33:43.542734312 - drop-icmp - start drop-icmp ci-op-jqc4wgr6-7cadb-wfmh5-worker-centralus1-xvccc'
      2023-02-13T11:33:43.543322816Z I0213 11:33:43.542734312 - drop-icmp - start drop-icmp ci-op-jqc4wgr6-7cadb-wfmh5-worker-centralus1-xvccc
      2023-02-13T11:33:43.543334916Z + iptables -X AZURE_CHECK_ICMP_SOURCE
      2023-02-13T11:33:43.550656359Z iptables v1.8.4 (nf_tables):  CHAIN_USER_DEL failed (Device or resource busy): chain AZURE_CHECK_ICMP_SOURCE
      2023-02-13T11:33:43.550990161Z + true
      2023-02-13T11:33:43.551027861Z + iptables -N AZURE_CHECK_ICMP_SOURCE
      2023-02-13T11:33:43.553045373Z iptables: Chain already exists.
      2023-02-13T11:33:43.553270874Z + true
      2023-02-13T11:33:43.553297674Z + iptables -F AZURE_CHECK_ICMP_SOURCE
      2023-02-13T11:33:43.555586788Z + iptables -D INPUT -p icmp --icmp-type fragmentation-needed -j AZURE_CHECK_ICMP_SOURCE
      2023-02-13T11:33:43.558080002Z + iptables -I INPUT -p icmp --icmp-type fragmentation-needed -j AZURE_CHECK_ICMP_SOURCE
      2023-02-13T11:33:43.565565646Z + iptables -N AZURE_ICMP_ACTION
      2023-02-13T11:33:43.567562758Z iptables: Chain already exists.
      2023-02-13T11:33:43.567818259Z + true
      2023-02-13T11:33:43.567851159Z + iptables -F AZURE_ICMP_ACTION
      2023-02-13T11:33:43.570072272Z + iptables -A AZURE_ICMP_ACTION -j LOG
      2023-02-13T11:33:43.576384010Z + iptables -A AZURE_ICMP_ACTION -j DROP
      2023-02-13T11:33:43.578980725Z + oc observe pods -n openshift-sdn --listen-addr= -l app=sdn -a '{ .status.hostIP }' -- /var/run/add_iptables.sh
      2023-02-13T11:33:43.579079025Z /bin/bash: line 30: oc: command not found
      

       

      Looks like it was caused by https://github.com/openshift/cluster-network-operator/pull/1681

       

      Attachments

        Issue Links

          Activity

            People

              stbenjam Stephen Benjamin
              stbenjam Stephen Benjamin
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: