Description of problem:

When a Feature is disabled in the CR, like route, localDB or monitoring, the Operator attempts to delete any resources that would have been created if this feature was enabled in the CR:

• https://github.com/redhat-developer/rhdh-operator/blob/release-1.9/internal/controller/backstage_controller.go#L190-L214
• https://github.com/redhat-developer/rhdh-operator/blob/release-1.9/internal/controller/monitor.go#L21-L28

The problem is that the user might have purposely created such resources, for example in an attempt to override some behavior that is not possible yet via the CR.

Noticed this for example when trying to use an external Database with the Orchestrator enabled, but still having the SonataflowPlatform resource managed by the Operator.. For this, we tried to manually create a DB Service named "backstage-psql-<backstage-name>" (and pointing to the external DB domain) and a DB Secret named "backstage-psql-secret-<backstage-name>". The Operator would automatically delete such resources upon reconciliation.

Prerequisites (if any, like setup, operators/versions):

Tried against main and release-1.9 branches.

Steps to Reproduce

Example with Route disabled, but we have the same issue with local DB and Monitoring features when they are disabled.

apiVersion: route.openshift.io/v1
kind: Route
metadata: 
  # This is the name of the resource the Operator would have created if the Route Feature was enabled in the CR
  name: backstage-bs-route-disabled
  labels: 
    com.example/label: "my-label"
spec: 
  path: /
  port: 
    targetPort: http-backend
  tls: 
    insecureEdgeTerminationPolicy: Redirect
    termination: edge
  to: 
    kind: Service
    name: backstage-bs-route-disabled
    weight: 100
  wildcardPolicy: None

---
apiVersion: rhdh.redhat.com/v1alpha5
kind: Backstage
metadata: 
  name: bs-route-disabled
spec: 
  application: 
    route: 
      enabled: false

Actual results:

Right after CR reconciliation, the Route "backstage-bs-route-disabled" is deleted from the cluster:

❯ oc get route
No resources found in my-ns namespace.

❯ oc get backstage            
NAME                AGE
bs-route-disabled   25m

❯ oc get po   
NAME                                           READY   STATUS    RESTARTS   AGE
backstage-bs-route-disabled-5b49b96576-tx6d5   1/1     Running   0          24m
backstage-psql-bs-route-disabled-0             1/1     Running   0          24m

Expected results:

We should be extremely careful with such resource deletion. Maybe not delete anything (but I guess this might be confusing), or make sure the resources we are deleting were really created by the Operator.

Reproducibility (Always/Intermittent/Only Once):

Always

Build Details:

Additional info (Such as Logs, Screenshots, etc):