Uploaded image for project: 'Hybrid Cloud Console'
  1. Hybrid Cloud Console
  2. RHCLOUD-24995

[QE] Compliance integrated tests fail on finding email content

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • Unset
    • No

      Note: Issue is related to iqe-notifications-plugin and the way how they are looking for notifications content.

      Some time ago Compliance tests started failing because email content can't be found. Our tests can be found here https://gitlab.cee.redhat.com/insights-qe/iqe-compliance-plugin/-/blob/master/iqe_compliance/tests/test_notifications_email.py .

      I believe it happens because there are too many events and there is a limitation to just 200 here https://gitlab.cee.redhat.com/insights-qe/iqe-notifications-plugin/-/blob/master/iqe_notifications/utils/notifications_utils.py#L413 . 

      So there is no pagination implemented and we take only 200 events. I tried to run it today and print these events - I see 31st January in some headers and today is 28th February. So looks like there is a huge amount of events but we take 200 from the end.

      My suggestions:
      1) Pagination ? - not sure if it worth if we could have a ton of these events
      2) Maybe we could take 200 but the most fresh by date (I don't really know how it works on backend side and if it's already sorted there)
      3) Fetch emails by datetime as Devidas mentioned.

      slack discussion is here https://redhat-internal.slack.com/archives/CU8TU5KPF/p1677590765536819 

      UPD:
      **
      I had some time to take a look on it. I found this doc about emails and content usage here https://developers.google.com/gmail/api/reference/rest/v1/users.messages/list . I tried to execute a small piece of iqe_notifications_plugin code here but with changed implementation a bit:
      result = service.users().messages().list(userId="me", labelIds=label_ids, q='Instant notification - Compliance - Red Hat Enterprise Linux',
      maxResults=500).execute()
      I specified q='Instant notification - Compliance...  just to reduce number of messages and get only Compliance notifications.
      Then I checked timestamps of first and last messages from results list
      First
      {'id': '1860821e117e4e95', 'threadId': '1860821e117e4e95', 'labelIds': ['UNREAD', 'Label_3658204369094513214', 'CATEGORY_UPDATES'], 'snippet': 'Red Hat logo
      System non-compliant Instant Notification - 31 Jan 2023 Your system, rhiqe.vkqdc8.tc3rl7.test, assigned to policy PCI-DSS v3.2.1 Control Baseline for Red Hat
      Enterprise Linux 7, has been', 'payload': {'partId': '', 'mimeType': 'text/html', 'filename': '', 'headers': [

      {'name': 'Delivered-To', 'value': 'insights-qe@gapps.redhat.com'}

      , {'name': 'Received', 'value': 'by 2002:a05:6a10:6785:b0:40a:a964:a354 with SMTP id p5csp184998pxu; Tue, 31 Jan 2023
      06:01:23 -0800 (PST)'}...
      Last
      service.users().messages().get(userId="me", id=w['messages'][499]["id"]).execute()

      {'id': '18589a252684efb0', 'threadId': '18589a252684efb0', 'labelIds': ['UNREAD', 'Label_3658204369094513214', 'CATEGORY_UPDATES'], 'snippet': 'Red Hat logo
      System non-compliant Instant Notification - 07 Jan 2023 Your system, rhiqe.ggati6.test, assigned to policy PCI-DSS v3.2.1 Control Baseline for Red Hat
      Enterprise Linux 7, has been marked', 'payload': {'partId': '', 'mimeType': 'text/html', 'filename': '', 'headers': [

      {'name': 'Delivered-To', 'value': 'insights-qe@gapps.redhat.com'}

      , {'name': 'Received', 'value': 'by 2002:a05:6a10:f4d6:b0:366:62e5:b0b9 with SMTP id q22csp1385917pxx; Fri, 6 Jan 2023
      16:29:54 -0800 (PST)'}.....
      As you can see timestamp of the first message is 31 Jan 2023  and 500x message shows 6 Jan 2023 timestamp.
      So it looks like from 1 February we stopped getting notifications on this account or something like that there are no fresh email notifications.
       

            rh-ee-dnoceda Diego Noceda
            eshamard Egor Shamardin
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: