Description of problem:

Collector pods get deleted after a couple of minutes and recreated. Setting CL instance to Unmanaged, the restarts stop.

Version-Release number of selected component (if applicable):

5.6.0

How reproducible:

Occurs in multiple clusters on customer side, not successfully reproduced in lab

Steps to Reproduce:

1. Upgraded from 5.5 to 5.6
2. Set CL instance to Managed...

Actual results:

Events show:
Tue Jan 24 09:31:00 CST 2023 openshift-logging Warning Invalid clusterlogforwarder/instance invalid: unrecognized outputs: [default], no valid outputs
Mon Jan 23 04:47:57 CST 2023 openshift-logging Warning Invalid clusterlogforwarder/instance

CLO logs only repeat:

{{}}

2023-01-19T17:28:03.202102402Z {"_ts":"2023-01-19T17:28:03.20110673Z","_level":"0","_component":"cluster-logging-operator","_message":"clusterlogging-controller error updating status","_error":{"msg":"Operation cannot be fulfilled on clusterloggings.logging.openshift.io \"instance\": the object has been modified; please apply your changes to the latest version and try again"}} 

CLF:

 
spec:
  outputs:
  - name: rsyslog-prod
    syslog:
      appName: example
      facility: user
      msgID: mymsg
      procID: myproc
      rfc: RFC5424
      severity: informational
    type: syslog
    url: udp://syslog.example.com:514
  pipelines:
  - inputRefs:
    - audit
    labels:
      syslog: example-prod
    name: syslog-prod
    outputRefs:
    - rsyslog-prod
    parse: json
  - inputRefs:
    - application
    - infrastructure
    name: enable-default-logs
    outputRefs:
    - default

When the CLO is set to Unmanaged, all logs flow normally to syslog and elasticsearch, so the configs themselves seem to work just fine.

Expected results:

I dont see any errors in the config; I would expect the CLO to either NOT restart pods, or else to log what it thinks the error is.

Additional info: