Description of Problem
The argocd-notification pod, which appears to be holding old TCP connections open to api.github.com. This results in a gradual increase in the number of open connections, potentially leading to a situation where all available ports are filled up, preventing further notifications from being sent.

Upstream Bug: https://github.com/argoproj/argo-cd/issues/22800

Additional Info

• <Any additional info such as logs, must-gather outputs, etc.>

Problem Reproduction

• <How do we reproduce the problem?>

Reproducibility

Always

Prerequisites/Environment

• Openshift GitOps 1.16.0

Steps to Reproduce

• Create a notification service for github service.
• Check the number of open connections by debug into pod

Expected Results

•  ArgoCD notification controller should properly close older connections. It should show only active connections. 

Actual Results

• ArgoCD notification controller not properly closing older connections. It's showing older connections too

Problem Analysis

• <Completed by engineering team as part of the triage/refinement process>

Root Cause

• <What is the root cause of the problem? Or, why is it not a bug?>

Workaround (If Possible)

Restarting the notification controller every so often:

oc scale deployment (...)-notifications-controller --replicas=0 
oc scale deployment (...)-notifications-controller --replicas=1

Fix Approaches

• <If we decide to fix this bug, how will we do it?>

Acceptance Criteria

• ...

Definition of Done

• Code Complete:
  • All code has been written, reviewed, and approved.
• Tested:
  • Unit tests have been written and passed.
  • Ensure code coverage is not reduced with the changes.
  • Integration tests have been automated.
  • System tests have been conducted, and all critical bugs have been fixed.
  • Tested and merged on OpenShift either upstream or downstream on a local build.
• Documentation:
  • User documentation or release notes have been written (if applicable).
• Build:
  • Code has been successfully built and integrated into the main repository / project.
  • Midstream changes (if applicable) are done, reviewed, approved and merged.
• Review:
  • Code has been peer-reviewed and meets coding standards.
  • All acceptance criteria defined in the user story have been met.
  • Tested by reviewer on OpenShift.
• Deployment:
  • The feature has been deployed on OpenShift cluster for testing.