To support temporary modifications to the kafka and other dependent resources, we will introduce a managedkafka.bf2.org/pause-reconcilation annotation. This will need to be recognized by sync as a fleetshard managed annotation as it's expected that this will be toggled manually and not set through the control plane. This will function similar to the strimzi annotation - when in place, the reconciliation will be skipped. Once removed, that update will trigger a reconciliation that will align to the desired state. The status will need to reflect that the instance is paused - potentially by returning ready unknown.

The most obvious side effect is that pausing will inhibit the instance from being upgraded. However it is not anticipated that instances will be paused for a long duration, so that is not a concern.

Additionally there should be a metric that identifies if the instance is paused.