Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-6791

Frequent unknown field warnings in Machine Config Operator logs

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Normal
    • None
    • 4.13.0
    • None
    • False
    • Hide

      None

      Show
      None

    Description

      Description of problem:

      The Machine Config Operator portion of the MCO repeatedly writes an abnormally large number of log entries warning about unknown fields being present:
      
      2023-01-27T20:59:25.860011687Z I0127 20:59:25.858182       1 start.go:47] Version: 4.13.0-0.ci.test-2023-01-27-203645-ci-op-55n3g3hw-latest (Raw: machine-config-daemon-4.6.0-202006240615.p0-1879-gb77691cb-dirty, Hash: b77691cb47242ba8a46e30d877778672c67d07a8)
      2023-01-27T20:59:25.921376648Z I0127 20:59:25.921337       1 leaderelection.go:248] attempting to acquire leader lease openshift-machine-config-operator/machine-config...
      2023-01-27T21:02:18.047949755Z I0127 21:02:18.047902       1 leaderelection.go:258] successfully acquired lease openshift-machine-config-operator/machine-config
      2023-01-27T21:02:18.402029704Z I0127 21:02:18.399996       1 operator.go:276] Starting MachineConfigOperator
      2023-01-27T21:02:20.573033163Z W0127 21:02:20.572968       1 warnings.go:70] unknown field "spec.dns.metadata.creationTimestamp"
      2023-01-27T21:02:20.573033163Z W0127 21:02:20.572984       1 warnings.go:70] unknown field "spec.dns.metadata.generation"
      2023-01-27T21:02:20.573033163Z W0127 21:02:20.572988       1 warnings.go:70] unknown field "spec.dns.metadata.managedFields"
      2023-01-27T21:02:20.573033163Z W0127 21:02:20.572990       1 warnings.go:70] unknown field "spec.dns.metadata.name"
      2023-01-27T21:02:20.573033163Z W0127 21:02:20.572993       1 warnings.go:70] unknown field "spec.dns.metadata.resourceVersion"
      2023-01-27T21:02:20.573033163Z W0127 21:02:20.572995       1 warnings.go:70] unknown field "spec.dns.metadata.uid"
      2023-01-27T21:02:20.573033163Z W0127 21:02:20.573025       1 warnings.go:70] unknown field "spec.infra.metadata.creationTimestamp"
      2023-01-27T21:02:20.573033163Z W0127 21:02:20.573029       1 warnings.go:70] unknown field "spec.infra.metadata.generation"
      2023-01-27T21:02:20.573084844Z W0127 21:02:20.573034       1 warnings.go:70] unknown field "spec.infra.metadata.managedFields"
      2023-01-27T21:02:20.573084844Z W0127 21:02:20.573037       1 warnings.go:70] unknown field "spec.infra.metadata.name"
      2023-01-27T21:02:20.573084844Z W0127 21:02:20.573039       1 warnings.go:70] unknown field "spec.infra.metadata.resourceVersion"
      2023-01-27T21:02:20.573084844Z W0127 21:02:20.573042       1 warnings.go:70] unknown field "spec.infra.metadata.uid"
      2023-01-27T21:02:26.565279362Z W0127 21:02:26.565245       1 warnings.go:70] unknown field "spec.dns.metadata.creationTimestamp"
      2023-01-27T21:02:26.565279362Z W0127 21:02:26.565261       1 warnings.go:70] unknown field "spec.dns.metadata.generation"
      2023-01-27T21:02:26.565279362Z W0127 21:02:26.565265       1 warnings.go:70] unknown field "spec.dns.metadata.managedFields"
      2023-01-27T21:02:26.565279362Z W0127 21:02:26.565268       1 warnings.go:70] unknown field "spec.dns.metadata.name"
      2023-01-27T21:02:26.565279362Z W0127 21:02:26.565271       1 warnings.go:70] unknown field "spec.dns.metadata.resourceVersion"
      2023-01-27T21:02:26.565279362Z W0127 21:02:26.565273       1 warnings.go:70] unknown field "spec.dns.metadata.uid"
      2023-01-27T21:02:26.565279362Z W0127 21:02:26.565276       1 warnings.go:70] unknown field "spec.infra.metadata.creationTimestamp"
      2023-01-27T21:02:26.565328323Z W0127 21:02:26.565279       1 warnings.go:70] unknown field "spec.infra.metadata.generation"
      2023-01-27T21:02:26.565328323Z W0127 21:02:26.565282       1 warnings.go:70] unknown field "spec.infra.metadata.managedFields"
      2023-01-27T21:02:26.565328323Z W0127 21:02:26.565285       1 warnings.go:70] unknown field "spec.infra.metadata.name"
      2023-01-27T21:02:26.565328323Z W0127 21:02:26.565289       1 warnings.go:70] unknown field "spec.infra.metadata.resourceVersion"
      2023-01-27T21:02:26.565328323Z W0127 21:02:26.565292       1 warnings.go:70] unknown field "spec.infra.metadata.uid"
      2023-01-27T21:02:33.003497308Z W0127 21:02:33.003463       1 warnings.go:70] unknown field "spec.dns.metadata.creationTimestamp"
      2023-01-27T21:02:33.003612650Z W0127 21:02:33.003603       1 warnings.go:70] unknown field "spec.dns.metadata.generation"
      2023-01-27T21:02:33.003633310Z W0127 21:02:33.003627       1 warnings.go:70] unknown field "spec.dns.metadata.managedFields"
      2023-01-27T21:02:33.003648951Z W0127 21:02:33.003643       1 warnings.go:70] unknown field "spec.dns.metadata.name"
      2023-01-27T21:02:33.003705222Z W0127 21:02:33.003698       1 warnings.go:70] unknown field "spec.dns.metadata.resourceVersion"
      2023-01-27T21:02:33.003722802Z W0127 21:02:33.003717       1 warnings.go:70] unknown field "spec.dns.metadata.uid"
      2023-01-27T21:02:33.003737862Z W0127 21:02:33.003732       1 warnings.go:70] unknown field "spec.infra.metadata.creationTimestamp"
      2023-01-27T21:02:33.003752543Z W0127 21:02:33.003747       1 warnings.go:70] unknown field "spec.infra.metadata.generation"
      2023-01-27T21:02:33.003780953Z W0127 21:02:33.003761       1 warnings.go:70] unknown field "spec.infra.metadata.managedFields"
      2023-01-27T21:02:33.003797094Z W0127 21:02:33.003791       1 warnings.go:70] unknown field "spec.infra.metadata.name"
      2023-01-27T21:02:33.003811044Z W0127 21:02:33.003806       1 warnings.go:70] unknown field "spec.infra.metadata.resourceVersion"
      2023-01-27T21:02:33.003825894Z W0127 21:02:33.003820       1 warnings.go:70] unknown field "spec.infra.metadata.uid"
      2023-01-27T21:02:39.171413281Z W0127 21:02:39.171368       1 warnings.go:70] unknown field "spec.dns.metadata.creationTimestamp"
      2023-01-27T21:02:39.171413281Z W0127 21:02:39.171382       1 warnings.go:70] unknown field "spec.dns.metadata.generation"
      2023-01-27T21:02:39.171413281Z W0127 21:02:39.171384       1 warnings.go:70] unknown field "spec.dns.metadata.managedFields"
      2023-01-27T21:02:39.171413281Z W0127 21:02:39.171387       1 warnings.go:70] unknown field "spec.dns.metadata.name"
      2023-01-27T21:02:39.171413281Z W0127 21:02:39.171390       1 warnings.go:70] unknown field "spec.dns.metadata.resourceVersion"
      2023-01-27T21:02:39.171413281Z W0127 21:02:39.171393       1 warnings.go:70] unknown field "spec.dns.metadata.uid"
      2023-01-27T21:02:39.171413281Z W0127 21:02:39.171396       1 warnings.go:70] unknown field "spec.infra.metadata.creationTimestamp"
      2023-01-27T21:02:39.171413281Z W0127 21:02:39.171400       1 warnings.go:70] unknown field "spec.infra.metadata.generation"
      2023-01-27T21:02:39.171413281Z W0127 21:02:39.171403       1 warnings.go:70] unknown field "spec.infra.metadata.managedFields"
      2023-01-27T21:02:39.171413281Z W0127 21:02:39.171406       1 warnings.go:70] unknown field "spec.infra.metadata.name"
      2023-01-27T21:02:39.171413281Z W0127 21:02:39.171409       1 warnings.go:70] unknown field "spec.infra.metadata.resourceVersion"
      2023-01-27T21:02:39.171452632Z W0127 21:02:39.171411       1 warnings.go:70] unknown field "spec.infra.metadata.uid"
      2023-01-27T21:02:49.169185063Z W0127 21:02:49.169140       1 warnings.go:70] unknown field "spec.dns.metadata.creationTimestamp"
      2023-01-27T21:02:49.169185063Z W0127 21:02:49.169156       1 warnings.go:70] unknown field "spec.dns.metadata.generation"
      2023-01-27T21:02:49.169185063Z W0127 21:02:49.169159       1 warnings.go:70] unknown field "spec.dns.metadata.managedFields"
      2023-01-27T21:02:49.169185063Z W0127 21:02:49.169160       1 warnings.go:70] unknown field "spec.dns.metadata.name"
      2023-01-27T21:02:49.169185063Z W0127 21:02:49.169162       1 warnings.go:70] unknown field "spec.dns.metadata.resourceVersion"
      2023-01-27T21:02:49.169185063Z W0127 21:02:49.169164       1 warnings.go:70] unknown field "spec.dns.metadata.uid"
      2023-01-27T21:02:49.169185063Z W0127 21:02:49.169165       1 warnings.go:70] unknown field "spec.infra.metadata.creationTimestamp"
      2023-01-27T21:02:49.169185063Z W0127 21:02:49.169167       1 warnings.go:70] unknown field "spec.infra.metadata.generation"
      2023-01-27T21:02:49.169185063Z W0127 21:02:49.169169       1 warnings.go:70] unknown field "spec.infra.metadata.managedFields"
      2023-01-27T21:02:49.169185063Z W0127 21:02:49.169170       1 warnings.go:70] unknown field "spec.infra.metadata.name"
      2023-01-27T21:02:49.169185063Z W0127 21:02:49.169172       1 warnings.go:70] unknown field "spec.infra.metadata.resourceVersion"
      2023-01-27T21:02:49.169185063Z W0127 21:02:49.169173       1 warnings.go:70] unknown field "spec.infra.metadata.uid"
      2023-01-27T21:03:01.461505881Z W0127 21:03:01.461423       1 warnings.go:70] unknown field "spec.dns.metadata.creationTimestamp"
      2023-01-27T21:03:01.461505881Z W0127 21:03:01.461471       1 warnings.go:70] unknown field "spec.dns.metadata.generation"
      2023-01-27T21:03:01.461505881Z W0127 21:03:01.461491       1 warnings.go:70] unknown field "spec.dns.metadata.managedFields"
      2023-01-27T21:03:01.461626674Z W0127 21:03:01.461510       1 warnings.go:70] unknown field "spec.dns.metadata.name"
      2023-01-27T21:03:01.461626674Z W0127 21:03:01.461525       1 warnings.go:70] unknown field "spec.dns.metadata.resourceVersion"
      2023-01-27T21:03:01.461626674Z W0127 21:03:01.461534       1 warnings.go:70] unknown field "spec.dns.metadata.uid"
      2023-01-27T21:03:01.461626674Z W0127 21:03:01.461542       1 warnings.go:70] unknown field "spec.infra.metadata.creationTimestamp"
      2023-01-27T21:03:01.461626674Z W0127 21:03:01.461549       1 warnings.go:70] unknown field "spec.infra.metadata.generation"
      2023-01-27T21:03:01.461626674Z W0127 21:03:01.461557       1 warnings.go:70] unknown field "spec.infra.metadata.managedFields"
      2023-01-27T21:03:01.461626674Z W0127 21:03:01.461565       1 warnings.go:70] unknown field "spec.infra.metadata.name"
      2023-01-27T21:03:01.461626674Z W0127 21:03:01.461572       1 warnings.go:70] unknown field "spec.infra.metadata.resourceVersion"
      2023-01-27T21:03:01.461626674Z W0127 21:03:01.461580       1 warnings.go:70] unknown field "spec.infra.metadata.uid"
      2023-01-27T21:03:07.977811118Z W0127 21:03:07.977760       1 warnings.go:70] unknown field "spec.dns.metadata.creationTimestamp"
      2023-01-27T21:03:07.977811118Z W0127 21:03:07.977776       1 warnings.go:70] unknown field "spec.dns.metadata.generation"
      2023-01-27T21:03:07.977811118Z W0127 21:03:07.977780       1 warnings.go:70] unknown field "spec.dns.metadata.managedFields"
      2023-01-27T21:03:07.977811118Z W0127 21:03:07.977783       1 warnings.go:70] unknown field "spec.dns.metadata.name"
      2023-01-27T21:03:07.977811118Z W0127 21:03:07.977785       1 warnings.go:70] unknown field "spec.dns.metadata.resourceVersion"
      2023-01-27T21:03:07.977811118Z W0127 21:03:07.977788       1 warnings.go:70] unknown field "spec.dns.metadata.uid"
      2023-01-27T21:03:07.977811118Z W0127 21:03:07.977791       1 warnings.go:70] unknown field "spec.infra.metadata.creationTimestamp"
      2023-01-27T21:03:07.977811118Z W0127 21:03:07.977794       1 warnings.go:70] unknown field "spec.infra.metadata.generation"
      2023-01-27T21:03:07.977811118Z W0127 21:03:07.977798       1 warnings.go:70] unknown field "spec.infra.metadata.managedFields"
      2023-01-27T21:03:07.977811118Z W0127 21:03:07.977801       1 warnings.go:70] unknown field "spec.infra.metadata.name"
      2023-01-27T21:03:07.977811118Z W0127 21:03:07.977804       1 warnings.go:70] unknown field "spec.infra.metadata.resourceVersion"
      2023-01-27T21:03:07.977811118Z W0127 21:03:07.977806       1 warnings.go:70] unknown field "spec.infra.metadata.uid"
      2023-01-27T21:03:14.520559778Z W0127 21:03:14.520519       1 warnings.go:70] unknown field "spec.dns.metadata.creationTimestamp"
      2023-01-27T21:03:14.520559778Z W0127 21:03:14.520540       1 warnings.go:70] unknown field "spec.dns.metadata.generation"
      2023-01-27T21:03:14.520559778Z W0127 21:03:14.520542       1 warnings.go:70] unknown field "spec.dns.metadata.managedFields"
      2023-01-27T21:03:14.520559778Z W0127 21:03:14.520544       1 warnings.go:70] unknown field "spec.dns.metadata.name"
      2023-01-27T21:03:14.520559778Z W0127 21:03:14.520546       1 warnings.go:70] unknown field "spec.dns.metadata.resourceVersion"
      2023-01-27T21:03:14.520559778Z W0127 21:03:14.520548       1 warnings.go:70] unknown field "spec.dns.metadata.uid"
      2023-01-27T21:03:14.520559778Z W0127 21:03:14.520549       1 warnings.go:70] unknown field "spec.infra.metadata.creationTimestamp"
      2023-01-27T21:03:14.520559778Z W0127 21:03:14.520551       1 warnings.go:70] unknown field "spec.infra.metadata.generation"
      2023-01-27T21:03:14.520559778Z W0127 21:03:14.520552       1 warnings.go:70] unknown field "spec.infra.metadata.managedFields"
      2023-01-27T21:03:14.520559778Z W0127 21:03:14.520554       1 warnings.go:70] unknown field "spec.infra.metadata.name"
      2023-01-27T21:03:14.520559778Z W0127 21:03:14.520555       1 warnings.go:70] unknown field "spec.infra.metadata.resourceVersion"
      2023-01-27T21:03:14.520601579Z W0127 21:03:14.520557       1 warnings.go:70] unknown field "spec.infra.metadata.uid"
      2023-01-27T21:03:20.739766364Z W0127 21:03:20.739732       1 warnings.go:70] unknown field "spec.dns.metadata.creationTimestamp"
      2023-01-27T21:03:20.739766364Z W0127 21:03:20.739748       1 warnings.go:70] unknown field "spec.dns.metadata.generation"
      2023-01-27T21:03:20.739766364Z W0127 21:03:20.739751       1 warnings.go:70] unknown field "spec.dns.metadata.managedFields"
      2023-01-27T21:03:20.739766364Z W0127 21:03:20.739753       1 warnings.go:70] unknown field "spec.dns.metadata.name"
      2023-01-27T21:03:20.739766364Z W0127 21:03:20.739754       1 warnings.go:70] unknown field "spec.dns.metadata.resourceVersion"
      2023-01-27T21:03:20.739766364Z W0127 21:03:20.739756       1 warnings.go:70] unknown field "spec.dns.metadata.uid"
      2023-01-27T21:03:20.739766364Z W0127 21:03:20.739757       1 warnings.go:70] unknown field "spec.infra.metadata.creationTimestamp"
      2023-01-27T21:03:20.739766364Z W0127 21:03:20.739759       1 warnings.go:70] unknown field "spec.infra.metadata.generation"
      2023-01-27T21:03:20.739766364Z W0127 21:03:20.739760       1 warnings.go:70] unknown field "spec.infra.metadata.managedFields"
      2023-01-27T21:03:20.739766364Z W0127 21:03:20.739762       1 warnings.go:70] unknown field "spec.infra.metadata.name"
      2023-01-27T21:03:20.739805474Z W0127 21:03:20.739764       1 warnings.go:70] unknown field "spec.infra.metadata.resourceVersion"
      2023-01-27T21:03:20.739805474Z W0127 21:03:20.739766       1 warnings.go:70] unknown field "spec.infra.metadata.uid"
      
      The MCO appears to be working correctly, but it is concerning that it is writing this many warning lines to its logs.

      Version-Release number of selected component (if applicable):

       

      How reproducible:

      Always

      Steps to Reproduce:

      1. Launch an OpenShift CI run.
      2. Review the MCO's pod logs: ($ oc logs -f pod/machine-config-operator-xxxx -n openshift-machine-config-operator)

      Expected results:

      I wouldn't have expected the MCO to write so many warnings to its log file.

      Additional info:

      The log seems to be coming from vendor/k8s.io/client-go/rest/warnings.go, indicating that the issue is occurring somewhere within the K8s stack. Judging from the unknown fields, my initial guess is that this has something to do with the ControllerConfig spec.

      Attachments

        Issue Links

          Activity

            People

              team-mco Team MCO
              zzlotnik@redhat.com Zack Zlotnik
              Rio Liu Rio Liu
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: