Uploaded image for project: 'OpenShift API for Data Protection'
  1. OpenShift API for Data Protection
  2. OADP-3156

(QE) Test automation issues and improvements

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • None
    • QE-Task
    • None
    • Quality / Stability / Reliability
    • 4
    • False
    • Hide

      None

      Show
      None
    • False
    • ToDo
    • 0
    • Very Likely
    • 0
    • None
    • Unset
    • Unknown
    • No

      The goal here is to go through our automation which have ~10% Fails due to bugs and some automation issues and try to:

      1. Fix the automation issues
      2. Skip cases which are marked by bug 
      3. Think of away to easy rerun failed tests only (nice to have)

          1.
          		 (QE) Think of a way to easily rerun failed test cases only Sub-task New Undefined Tareq Alayan  
          2.
          		 (QE) Add pre-run cleanup on BeforeSuite Spec Sub-task Closed Undefined Tareq Alayan  
          3.
          		 (QE) Add data integrity check to validate.yml tasks Sub-task Closed Normal Unassigned  
          4.
          		 (QE) Automate DataMover backup and restore after delete VScontent Sub-task New Critical Unassigned  
          5.
          		 (QE) Add Template for minio/MCG Sub-task Closed Undefined Maya Anilson (Inactive)  
          6.
          		 (QE) Add log message/warning that a bug was found but it resolved, Sub-task New Undefined Tareq Alayan  
          7.
          		 (QE) Add fix version check in Jira automation Sub-task New Undefined Unassigned  
          8.
          		 (QE) Remove ImageStream cases Sub-task Closed Major Prasad Joshi  
          9.
          		 (QE) VSL specs are not getting generated for AWS-MCG runs Sub-task Closed Major Shahaf Bahar (Inactive)  
          10.
          		 (QE) Reduce backup timeout once OADP-2038 is fixed Sub-task New Undefined Prasad Joshi  
          11.
          		 (QE) Investigate OADP-310 failure Sub-task Closed Major Shahaf Bahar (Inactive)  
          12.
          		 (QE) review skip logic for vsl labeled tests Sub-task Closed Undefined Tareq Alayan  
          13.
          		 (QE) Django application tests not working with disconnected cluster Sub-task Closed Undefined Amos Mastbaum  
          14.
          		 (QE) Add test case id for all automated data mover cases Sub-task Closed Major Prasad Joshi  
          15.
          		 (QE) Investigate OADP-222 failure Sub-task Closed Major Prasad Joshi  
          16.
          		 (QE) Fix Upgrade AfterSuite Sub-task Closed Undefined Amos Mastbaum  
          17.
          		 (QE) Use unique namespace in hooks suite Sub-task New Major Prasad Joshi  
          18.
          		 (QE) Add Velero error logs on scheduale tests fail Sub-task Closed Undefined Amos Mastbaum  
          19.
          		 (QE) Collect MCG logs Sub-task Closed Undefined Amos Mastbaum  
          20.
          		 (QE) Add validation for verifying backupRepositories type, PVB and PVR for kopia and restic cases Sub-task Closed Undefined Prasad Joshi  
          21.
          		 (QE) Ensure all the additional resources that gets generated during the dataMover operation is removed Sub-task New Undefined Prasad Joshi  
          22.
          		 (QE) Django application failing on SNO cluster Sub-task Closed Undefined Tareq Alayan  
          23.
          		 (QE) Cassandra application failing on SNO cluster Sub-task Closed Undefined Tareq Alayan  
          24.
          		 (QE) Mysql application failing on SNO cluster Sub-task Closed Undefined Tareq Alayan  
          25.
          		 (QE) VSL spec in DPA not generated in SNO cluster Sub-task Closed Undefined Shahaf Bahar (Inactive)  
          26.
          		 (QE) SNO cluster deployed with 2 default SCs Sub-task Closed Undefined Tareq Alayan  
          27.
          		 (QE) Support running tests (not only dry run) in ci pipline and other improvements Sub-task New Undefined Prasad Joshi  
          28.
          		 (QE) Fix NodeAgent helper function Sub-task Closed Undefined Prasad Joshi  
          29.
          		 (QE) Skip all node selector tests on SNO cluster Sub-task Closed Undefined Shahaf Bahar (Inactive)  
          30.
          		 (QE) Add additional validation for CSI Sub-task Closed Critical Prasad Joshi  
          31.
          		 (QE) Investigate and fix automated case OADP-498 Sub-task Closed Major Prasad Joshi  
          32.
          		 (QE) Investigate and fix OADP-404/OADP-457 failure Sub-task Closed Major Prasad Joshi  
          33.
          		 (QE) Investigate and fix OADP-381 failure Sub-task Closed Undefined Prasad Joshi  
          34.
          		 (QE) Investigate and run Disconnected test run on 1.3.1 build Sub-task Closed Undefined Amos Mastbaum  
          35.
          		 (QE) Adding SNO to test matrix & Execute regression Sub-task Closed Undefined Prasad Joshi  
          36.
          		 (QE) Investigate ocp-mysql failure with azurefile-csi SC Sub-task New Undefined Prasad Joshi  
          37.
          		 (QE) Add new label for interop cases Sub-task Closed Critical Prasad Joshi  
          38.
          		 (QE) Fix cloudStorage suite Sub-task Closed Critical Prasad Joshi  
          39.
          		 Modify ROSA provisioning pipeline to support both CCO and Non CCO worklow Sub-task New Undefined Tareq Alayan  
          40.
          		 Execute ROSA STS regression runs for OCP 4.14 & 4.15 Sub-task Closed Undefined Prasad Joshi  
          41.
          		 (QE) Fix default test filter logic Sub-task New Undefined Unassigned  
          42.
          		 (QE) Add pod validations logic in ocp-imagestreams app role Sub-task Closed Undefined Prasad Joshi  
          43.
          		 (QE) Investigate and fix skipping dataMover case OADP-274 Sub-task Closed Major Prasad Joshi  
          44.
          		 (QE) Update OADP-284 case to include multiple namespaces Sub-task Closed Undefined Prasad Joshi  
          45.
          		 (QE) Fix storageClass issue Sub-task Closed Critical Prasad Joshi  
          46.
          		 (QE) Fix unsupportedOverrides cases for ARM, P and Z architecture Sub-task Closed Major Prasad Joshi  
          47.
          		 (QE) Remove the bug-id label from the S3url port cases Sub-task Closed Normal Prasad Joshi  
          48.
          		 (QE) Fix exclude resources cases Sub-task Closed Major Prasad Joshi  
          49.
          		 (QE) Fix automated case OADP-222 Sub-task New Undefined Unassigned  
          50.
          		 (QE) Fix ocp-nginxpv role validation Sub-task Closed Undefined Prasad Joshi  
          51.
          		 (QE) Add functionality to run cross cluster backup and restore scenario Sub-task Closed Critical Prasad Joshi  
          52.
          		 (QE) Fix GCP WIF deployment issue Sub-task Closed Undefined Prasad Joshi  
          53.
          		 (QE) Increase the format.MaxLength to 8000 Sub-task Closed Undefined Prasad Joshi  
          54.
          		 (QE) Fix app deployment issue for release-1.0 branch Sub-task Closed Undefined Prasad Joshi  
          55.
          		 (QE) Skip VSL related cases when the cloud provider value is gcp-wif or ibmcloud Sub-task Closed Undefined Prasad Joshi  
          56.
          		 (QE) Investigate OADP-458 with cephfs Sub-task Closed Undefined Sachin Singla  
          57.
          		 (QE) Add dummy data into mysql PVC Sub-task Closed Undefined Prasad Joshi  
          58.
          		 (QE) Investigate OADP-492 failure with cephfs Sub-task Closed Undefined Sachin Singla  
          59.
          		 (QE) FIx OADP-363 Sub-task New Undefined Sachin Singla  
          60.
          		 (QE) FIx OADP-363 Sub-task Closed Undefined Tareq Alayan  
          61.
          		 (QE) Skip the PVR validation for empty PVC Sub-task Closed Undefined Prasad Joshi  
          62.
          		 (QE) Investigate OADP-458 & 492 with azure-arm Sub-task Closed Undefined Sachin Singla  
          63.
          		 Investigate OADP-94 with azure-arm Sub-task Closed Undefined Sachin Singla  
          64.
          		 Add option to install CNV via OADP pipeline Sub-task Closed Normal Tareq Alayan  
          65.
          		 Investigate log upload issue with aws-ocs-cephfs run Sub-task New Undefined Tareq Alayan  
          66.
          		 (QE) Add an empty pvc app in ocpdeployer Sub-task New Undefined Unassigned  
          67.
          		 (QE) Add polarion cases for empty pvc application. Sub-task New Undefined Prasad Joshi  
          68.
          		 (QE) Investigate and fix the OADP-509 failure Sub-task Closed Undefined Prasad Joshi  
          69.
          		 (QE) Make sure smoke label includes all the important cases Sub-task Closed Undefined Prasad Joshi  
          70.
          		 (QE) Investigate and fix OADP-413 Sub-task Closed Undefined Prasad Joshi  
          71.
          		 (QE) Add CSI label for cases, like datamover Sub-task New Undefined Sachin Singla  
          72.
          		 (QE) Update the ocp-mysql role to work with namespace admin user Sub-task Closed Critical Prasad Joshi  
          73.
          		 (QE) Fix OADP-404 failure Sub-task Closed Undefined Prasad Joshi  
          74.
          		 (QE) Add support for running tests on velero upstream Sub-task Closed Undefined Prasad Joshi  
          75.
          		 (QE) Investigate and fix include/exclude resource cases Sub-task Closed Undefined Prasad Joshi  
          76.
          		 (QE) Push multi-arch image for velero-plugin-for-aws in oadp-qe repo Sub-task Closed Undefined Prasad Joshi  
          77.
          		 (QE) Fix ocp-imagestream application deployment Sub-task Closed Undefined Prasad Joshi  
          78.
          		 (QE) Investigate and fix IBM-z stream interop tests Sub-task Closed Undefined Prasad Joshi  
          79.
          		 (QE) Update go mod dependencies Sub-task Closed Critical Prasad Joshi  
          80.
          		 (QE) Install Upstream Velero in Multiple namespaces using test pipeline Sub-task New Undefined Tareq Alayan  
          81.
          		 (QE) Remove CSI plugin related validation related cases from test suite Sub-task Closed Undefined Prasad Joshi  
          82.
          		 (QE) Investigate and fix oadp-530 issue Sub-task Closed Undefined Prasad Joshi  
          83.
          		 (QE) Stabilize ocp-todolist-mongodb-block application validation Sub-task Closed Undefined Prasad Joshi  
          84.
          		 (QE) Fix automated case oadp-420 Sub-task Closed Undefined Prasad Joshi  
          85.
          		 (QE) Fix dpa panic issue Sub-task Closed Undefined Prasad Joshi  
          86.
          		 (QE) Add check for leader election in e2e automation Sub-task Closed Undefined Prasad Joshi  
          87.
          		 (QE) Update the CSI plugin image to sha format Sub-task Closed Undefined Prasad Joshi  
          88.
          		 (QE) Fix image issue in disconnect cluster Sub-task Closed Major Prasad Joshi  
          89.
          		 (QE) Fix automated case OADP-363 Sub-task Closed Undefined Prasad Joshi  
          90.
          		 (QE) Verify the bug OADP-3843 is fixed in OADP 1.4 and remove the bug-id label from the automated test Sub-task Closed Undefined Prasad Joshi  
          91.
          		 (QE) Fix apps deployer issue for AWS gov cloud Sub-task Closed Blocker M Sajid Mansoori  
          92.
          		 (QE) Fix kubevirt ansible issue Sub-task Closed Major Prasad Joshi  
          93.
          		 (QE) Investigate and fix oadp must gather failure in case of s3url is specified Sub-task Closed Undefined Prasad Joshi  
          94.
          		 (QE) Fix automated case OADP-560 Sub-task Closed Undefined Prasad Joshi  
          95.
          		 (QE) Remove bug-id label from all the non relevant test cases Sub-task Release Pending Undefined Prasad Joshi  
          96.
          		 (QE) Bump up oadp operator dependency Sub-task Closed Undefined Prasad Joshi  
          97.
          		 (QE) Automate the test case for oadp-467 Sub-task Closed Undefined Ukthi Prasad

          100%
          Original Estimate - 0 minutes Original Estimate - 0 minutes
          Time Spent - 3 weeks
          98.
          		 (QE)Automate the Test case for oadp-522 Sub-task Closed Undefined Ukthi Prasad

          100%
          Original Estimate - 0 minutes Original Estimate - 0 minutes
          Time Spent - 24 minutes
          99.
          		 (QE) Automate test case for oadp-94 Sub-task Closed Undefined Ukthi Prasad

          100%
          Original Estimate - 0 minutes Original Estimate - 0 minutes
          Time Spent - 3 weeks
          100.
          		 (QE) Automating test case for oadp-406 Sub-task Closed Undefined Ukthi Prasad

          100%
          Original Estimate - 0 minutes Original Estimate - 0 minutes
          Time Spent - 3 weeks
          101.
          		 (QE) Automating for oadp-394 Sub-task Closed Undefined Ukthi Prasad

          100%
          Original Estimate - 0 minutes Original Estimate - 0 minutes
          Time Spent - 3 weeks
          102.
          		 Automate Backup Encryption cases SSE-C & SSE-KMS Sub-task Closed Undefined Sachin Singla  
          103.
          		 (QE) Automate Parallel file upload case Sub-task Closed Undefined Sachin Singla  
          104.
          		 Automate oadp-394 Sub-task Closed Undefined Sachin Singla  
          105.
          		 (QE) Automate resource filtering with CSI cases Sub-task Closed Undefined Sachin Singla  
          106.
          		 (QE) Automate oadp-599 Sub-task Closed Undefined Sachin Singla  
          107.
          		 (QE) Automate oadp-579 Sub-task Closed Undefined Sachin Singla  
          108.
          		 (QE) Automate oadp 605 & 606 Sub-task Closed Undefined Sachin Singla  
          109.
          		 (QE) automate oadp 423 & 426 Sub-task Closed Undefined Sachin Singla  
          110.
          		 (QE) Automate oadp-535, 536, 545 Sub-task Closed Undefined Tareq Alayan  
          111.
          		 (QE) Automate oadp-548 Sub-task Closed Undefined Tareq Alayan  
          112.
          		 Automate cross cluster dm cases - oadp-583 , 586, 596 Sub-task Closed Undefined Sachin Singla  
          113.
          		 (QE) Automate Storage class mapping cases for cross cluster test runs Sub-task Closed Undefined Tareq Alayan  
          114.
          		 (QE) Automate oadp608, 609 Sub-task Closed Undefined Sachin Singla  
          115.
          		 (QE) Add cleanup logic for deleting backupRepository at the end of LRT test Sub-task In Progress Undefined Prasad Joshi  
          116.
          		 (QE) Automate the test case 542 & 536 Sub-task Closed Undefined Ukthi Prasad

          100%
          Original Estimate - 0 minutes Original Estimate - 0 minutes
          Time Spent - 3 weeks
          117.
          		 (QE) Add validation for verifying datamover related CR's Sub-task Closed Undefined Prasad Joshi  

              talayan@redhat.com Tareq Alayan
              talayan@redhat.com Tareq Alayan
              Sachin Singla Sachin Singla
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 15 weeks, 24 minutes
                  15w 24m