On Oct 20th, AWS had a major outage in us-east-1. During this outage, IAM services returned internal errors.

During this time, hypershift_cluster_invalid_aws_creds metric was set to "1", meaning the credentials are invalid. This is not accurate.

SRE uses this metric to automate notifications to customers and disable alerting for clusters. 

    N/A

    100%

    1. Set ValidAWSCredentials to Unknown to simulate AWS returning 500s / healthchecks not working
    2. Check metric state
    3.
    

    Metric is set to 1 (invalid credentials) for unknown states

    Metric should distinguish between true/false/unknown