PO/Tech Lead NOTE: Developer Pairing Opportunity

Implement and configure Snowpipe pipelines for prod through production environments to automate data ingestion from S3 buckets into Snowflake.

Create external stage for data product.

Promote Snowpipe migration file and source binding to prod:
 Example dev pr: https://gitlab.cee.redhat.com/dataverse/dataverse-config/dataproduct-config/-/merge_requests/5237/diffs 

Create Pipes for each s3 bucket:

• ocp_cluster_data
• sku_product_tag
  _ subscriptions _This is the floorist export
• tally

(not yet implemented in floorist)

• tally_snapshots
• tally_measurements
  _ subscription _This is the raw table
• subscription_measurements
• hosts
• host_tally_buckets
•  

Acceptance Criteria:

• For environments prod
• When new data is added to the s3 buckets, then the snowflake stages are populated.
• Snowflake Snowpipe DB- We need to verify record count matches Floorist export