Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-1662

mcd_update_state metric should have a single time-series per node

    XMLWordPrintable

Details

    • Moderate
    • MCO Sprint 232, MCO Sprint 233
    • 2
    • False
    • Hide

      None

      Show
      None
    • Cleaned up MCD state metrics
    • Bug Fix

    Description

      Seen in build02, currently running 4.12.0-ec.3:

      mcd_update_state{node="build0-gstfj-m-0.c.openshift-ci-build-farm.internal"}
      

      returns:

      Those are identical, except:

      • The first has config populated with rendered-... and has a non-zero value.
      • The second has config empty and has a zero value.

      Looking at the backing code, my guess is that we're doing something like this:

      • Things are happy; export with a populated config.
      • Things get sad. Export with an empty config and a new error. But the happy time-series sticks around, and somehow has the value move to zero.
      • Things get happy again; and we return to setting a value for the happy time-series. But the sad time-series sticks around, and somehow has the value move to zero.

      Or something like that.  I expect we want to drop the zero-valued time-series, but I'm not clear enough on how the MCO pushes values into the export set to have code suggestions.

       

      Attachments

        1. image-2022-09-22-18-11-46-852.png
          86 kB
          W. Trevor King
        2. image-2022-09-22-18-14-03-536.png
          64 kB
          W. Trevor King
        3. image-2022-09-22-18-18-19-319.png
          131 kB
          W. Trevor King
        4. image-2022-09-22-18-22-08-321.png
          183 kB
          W. Trevor King

        Issue Links

          Activity

            People

              djoshy David Joshy
              trking W. Trevor King
              Sergio Regidor de la Rosa Sergio Regidor de la Rosa
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: