# deployed 1.2.0-10 Operator bundle: $ netobserv_main.sh install Now using project "netobserv" on server "https://api.memodi-02160921.qe.devcluster.openshift.com:6443". You can add applications to this project with the 'new-app' command. For example, try: oc new-app rails-postgresql-example to build a new example application in Ruby. Or use kubectl to deploy a simple Kubernetes application: kubectl create deployment hello-node --image=k8s.gcr.io/e2e-test-images/agnhost:2.33 -- /agnhost serve-hostname operatorgroup.operators.coreos.com/netobserv created ====> Using most recent downstream catalog bundle ====> Using catalog image quay.io/openshift-qe-optional-operators/aosqe-index:v4.12 catalogsource.operators.coreos.com/qe-app-registry created imagecontentsourcepolicy.operator.openshift.io/brew-registry created pod/qe-app-registry-4cqwv condition met subscription.operators.coreos.com/netobserv-operator created pod/netobserv-controller-manager-65f867b4b8-r9jrq condition met NAME CREATED AT flowcollectors.flows.netobserv.io 2023-02-16T15:28:02Z flowcollector.flows.netobserv.io/cluster created ====> Waiting for flowlogs-pipeline pod to be ready NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE flowlogs-pipeline 0 0 0 0 0 0s pod/flowlogs-pipeline-5glr4 condition met pod/flowlogs-pipeline-gszw5 condition met pod/flowlogs-pipeline-rb4rl condition met pod/flowlogs-pipeline-rmdrt condition met pod/flowlogs-pipeline-vtvfz condition met pod/flowlogs-pipeline-z9r8q condition met persistentvolumeclaim/loki-store created configmap/loki-config created pod/loki created service/loki created # deployed kafka (⎈ |memodi-02160921:netobserv)memodi@memodi-mac:/Users/memodi/workspaces/repos/ocp-qe-perfscale-ci/scripts (netobserv-perf-tests $=) $ source env.sh ====> Deploying Kafka Warning: resource namespaces/netobserv is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by oc apply. oc apply should only be used on resources created declaratively by either oc create --save-config or oc apply. The missing annotation will be patched automatically. namespace/netobserv configured ====> Creating amq-streams Subscription subscription.operators.coreos.com/amq-streams created pod/amq-streams-cluster-operator-v2.3.0-1-54cfd5c95c-8bcpf condition met ====> Creating kafka-metrics ConfigMap and kafka-resources-metrics PodMonitor configmap/kafka-metrics created podmonitor.monitoring.coreos.com/kafka-resources-metrics created ====> Creating kafka-cluster Kafka kafka.kafka.strimzi.io/kafka-cluster created ====> Creating network-flows KafkaTopic kafkatopic.kafka.strimzi.io/network-flows created kafkatopic.kafka.strimzi.io/network-flows condition met ====> Update flowcollector to use KAFKA deploymentModel flowcollector.flows.netobserv.io/cluster patched ====> Update flowcollector replicas flowcollector.flows.netobserv.io/cluster patched (no change) ====> Update clusterrolebinding Service Account from flowlogs-pipeline to flowlogs-pipeline-transformer clusterrole.rbac.authorization.k8s.io/lokistack-netobserv-tenant-logs created clusterrolebinding.rbac.authorization.k8s.io/lokistack-netobserv-tenant-logs created flowcollector.flows.netobserv.io/cluster condition met # reproduced the issue: $ oc get flowcollector -w NAME AGENT SAMPLING (EBPF) DEPLOYMENT MODEL STATUS cluster EBPF 50 KAFKA ReconcileFLPFailed cluster EBPF 50 KAFKA Ready cluster EBPF 50 KAFKA ReconcileFLPFailed cluster EBPF 50 KAFKA Ready cluster EBPF 50 KAFKA ReconcileFLPFailed cluster EBPF 50 KAFKA Ready cluster EBPF 50 KAFKA ReconcileFLPFailed cluster EBPF 50 KAFKA Ready cluster EBPF 50 KAFKA ReconcileFLPFailed cluster EBPF 50 KAFKA Ready cluster EBPF 50 KAFKA ReconcileFLPFailed cluster EBPF 50 KAFKA Ready cluster EBPF 50 KAFKA ReconcileFLPFailed cluster EBPF 50 KAFKA Ready cluster EBPF 50 KAFKA ReconcileFLPFailed # Uninstalled 1.2.0-10 Operator while keeping flowcollector components intact: Checked out the PR and deployed NOO pre-merge image: (⎈ |memodi-02160921:netobserv)memodi@memodi-mac:/Users/memodi/workspaces/repos/netobserv/network-observability-operator (release-1.2 $=) $ gh pr checkout 270 $ VERSION=51c35f0 make deploy /Users/memodi/workspaces/repos/netobserv/network-observability-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..." /Users/memodi/workspaces/repos/netobserv/network-observability-operator/bin/controller-gen "crd:trivialVersions=true,preserveUnknownFields=false" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases /Users/memodi/workspaces/repos/netobserv/network-observability-operator/bin/crdoc --resources config/crd/bases/flows.netobserv.io_flowcollectors.yaml --output docs/FlowCollector.md cd config/manager && /Users/memodi/workspaces/repos/netobserv/network-observability-operator/bin/kustomize edit set image controller=quay.io/netobserv/network-observability-operator:51c35f0 sed -i -r 's~ebpf-agent:.+~ebpf-agent:main~' ./config/manager/manager.yaml sed -i -r 's~flowlogs-pipeline:.+~flowlogs-pipeline:main~' ./config/manager/manager.yaml sed -i -r 's~console-plugin:.+~console-plugin:main~' ./config/manager/manager.yaml /Users/memodi/workspaces/repos/netobserv/network-observability-operator/bin/kustomize build config/default | kubectl apply -f - namespace/netobserv configured Warning: resource customresourcedefinitions/flowcollectors.flows.netobserv.io is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically. customresourcedefinition.apiextensions.k8s.io/flowcollectors.flows.netobserv.io configured Warning: resource serviceaccounts/netobserv-controller-manager is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically. serviceaccount/netobserv-controller-manager configured role.rbac.authorization.k8s.io/netobserv-leader-election-role created clusterrole.rbac.authorization.k8s.io/netobserv-manager-role created Warning: resource clusterroles/netobserv-metrics-reader is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically. clusterrole.rbac.authorization.k8s.io/netobserv-metrics-reader configured clusterrole.rbac.authorization.k8s.io/netobserv-proxy-role created rolebinding.rbac.authorization.k8s.io/netobserv-leader-election-rolebinding created clusterrolebinding.rbac.authorization.k8s.io/netobserv-manager-rolebinding created clusterrolebinding.rbac.authorization.k8s.io/netobserv-proxy-rolebinding created configmap/netobserv-manager-config created service/netobserv-metrics-service created deployment.apps/netobserv-controller-manager created $ oc describe pod/netobserv-controller-manager-c88d9db76-kkhd9 | egrep Image: Image: quay.io/netobserv/network-observability-operator:51c35f0 Image: gcr.io/kubebuilder/kube-rbac-proxy:v0.8.0 Verified issue is fixed - it no longer goes into ReconsoleFLPFailed error: $ oc get flowcollector -w NAME AGENT SAMPLING (EBPF) DEPLOYMENT MODEL STATUS cluster EBPF 50 KAFKA ReconcileFLPFailed cluster EBPF 50 KAFKA Ready No errors are seen in NOO pod logs. # also verified DIRECT deploymendModel - flow logs are being processed fine. $ oc get flowcollector -o jsonpath='{.items[].spec.deploymentModel}' DIRECT cluster EBPF 50 KAFKA Ready cluster EBPF 50 DIRECT Ready