1. 1. Partially cloned from 17.1 bug 2213126 ##

Description of problem:
Security group logging RFE test plan for OpenStack passes except a single test - Verify rate limit and burst size enforcement for stateful groups:

Test description:
Rate limit specifies the amount of packets per second that can be transferred to the OVN controller to be logged.
Burst limit is the size of queue when rate limit is fully used.
Need to make sure that both parameters are enforced correctly.

Notice that fair meter should not affect the result, each ACL has individual limits of rate and burst.

Version-Release number of selected component (if applicable):
ovn21.12
RHOS-16.2

How reproducible:
Every time.

Steps to Reproduce:
Setup for Test Steps:

Create new security group

1. SG=`openstack security group create sg_test_rate_limit -f value -c id`
2. openstack security group rule create --ingress --protocol icmp $SG

Create new virtual machine

1. curl -L http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img --output cirros-0.4.0-x86_64-disk.img
2. openstack image create --file /home/stack/cirros-0.4.0-x86_64-disk.img cirros-0.4.0-x86_64-disk.img
3. openstack flavor create --ram 128 --vcpus 1 m1.micro
4. openstack network create net_1
5. openstack subnet create --subnet-range 10.100.0.0/24 --network net_1 subnet_1
6. openstack router create router_1
7. openstack router set --external-gateway nova router_1
8. openstack router add subnet router_1 subnet_1
9. openstack server create --image cirros-0.4.0-x86_64-disk.img --flavor m1.micro --network net_1 --security-group $SG --wait vm_1
10. openstack floating ip create --floating-ip-address 10.0.0.201 nova
11. openstack server add floating ip vm_1 10.0.0.201

Enable logging for the newly created security group:

1. openstack network log create --resource-type security_group --resource $SG --event ALL test_log

Make sure that VM is reachable from the undercloud host:

1. ping 10.0.0.201

Get the IP address of the VM’s hypervisor:

1. CIP=`openstack server show vm_1 -f value -c OS-EXT-SRV-ATTR:hypervisor_hostname | awk -F "." ' {print $1}' | xargs -I {} grep {} /etc/hosts | awk '/ctlplane/ {print $1}

   '`

Ensure default values of rate and burst limit are applied

1. . ~/stackrc && ANSIBLE_DEPRECATION_WARNINGS=false ansible -b -i ~/overcloud-deploy/overcloud/tripleo-ansible-inventory.yaml -m shell -a \
   "crudini --set /var/lib/config-data/puppet-generated/neutron/etc/neutron/plugins/ml2/ml2_conf.ini network_log rate_limit 100" Controller
2. . ~/stackrc && ANSIBLE_DEPRECATION_WARNINGS=false ansible -b -i ~/overcloud-deploy/overcloud/tripleo-ansible-inventory.yaml -m shell -a \
   "crudini --set /var/lib/config-data/puppet-generated/neutron/etc/neutron/plugins/ml2/ml2_conf.ini network_log burst_limit 20" Controller
3. . ~/stackrc && ANSIBLE_DEPRECATION_WARNINGS=false ansible -b -i ~/overcloud-deploy/overcloud/tripleo-ansible-inventory.yaml -m shell -a \
   "podman restart neutron_api" Controller
4. sleep 10 && . ~/stackrc && ANSIBLE_DEPRECATION_WARNINGS=false ansible -b -i ~/overcloud-deploy/overcloud/tripleo-ansible-inventory.yaml -m shell -a \
   "podman exec ovn_controller ovn-nbctl list meter-band" Controller

Test Steps:

Step: Get the last available log entry ID number.

1. C1=$(ssh tripleo-admin@$CIP sudo grep acl_log /var/log/containers/stdouts/ovn_controller.log | tail -n1 | cut -d "|" -f 2); echo $C1
   Result: Log ID is a number with leading zeros that indicates amount of logs are generated on the specific compute node

Step: Send big amount of ICMP requests from the undercloud node for less than a second:

1. sudo ping 10.0.0.201 -i 0.0005 -c 400 | tail -n4
   Result: No traffic loss should be observed

Step: Check the last log entry ID and calculate the amount of logs generated from the last time:

1. C2=$(ssh tripleo-admin@$CIP sudo grep acl_log /var/log/containers/stdouts/ovn_controller.log | tail -n1 | cut -d "|" -f 2); echo $C2
   Result: Log amount (C2 - C1) should be almost equal (~10% difference) to the rate_limit + burst_limit.
   Result in between 112 and 138, according to default limit values.

Step: Send ICMP requests for a minute with packet rate higher than configured `rate_limit`:

1. sudo ping 10.0.0.201 -i 0.005 -c 12000 | tail -n4
   Result: No traffic loss should be observed

Step: Check the last log entry ID and so calculate the amount of logs generated from the last time:

1. C3=$(ssh tripleo-admin@$CIP sudo grep acl_log /var/log/containers/stdouts/ovn_controller.log | tail -n1 | cut -d "|" -f 2); echo $C3
   Result: Log amount (C3 - C2) should be almost equal to the rate_limit * 60 + burst_limit (~2% difference).
   Result in between 5900 and 6140, according to default limit values.

Actual results:
The first check fails (check under 1 second, burst+rate limits) - results in 33 packets logged, every test run result is 1/3 of expected value.
The second check passes every time (check over 1 minute).

Expected results:
Both checks should pass, if burst and rate limits not enforced as expected, maybe there is a regression.

1. 1. ##

I have created this bug to keep track in Neutron of the core OVN BZ I filed to get more insight on the inconsistencies with burst and rate limiting in OVN.