-
Bug
-
Resolution: Done-Errata
-
Critical
-
None
-
ACM 2.9.0, ACM 2.8.0
Description of problem:
On OCP 4.14 clusters being configured for ODF RDR, VolSync operator on managed clusters fails with below error in UI:
Operator failed install failed: deployment volsync-controller-manager not ready before timeout: deployment "volsync-controller-manager" exceeded its progress deadline
And it was observed that volsync-controller-manager pod is in CrashLoopBackOff state
$ oc get csv,pod -n openshift-operators NAME DISPLAY VERSION REPLACES PHASE clusterserviceversion.operators.coreos.com/odr-cluster-operator.v4.14.0-99.stable Openshift DR Cluster Operator 4.14.0-99.stable Succeeded clusterserviceversion.operators.coreos.com/volsync-product.v0.7.3 VolSync 0.7.3 volsync-product.v0.7.2 Failed NAME READY STATUS RESTARTS AGE pod/volsync-controller-manager-6d8f7b5d76-2ntbx 1/2 CrashLoopBackOff 7 (3m58s ago) 14m
Version-Release number of selected component (if applicable):
OCP: 4.14.0-0.nightly-2023-07-31-181848
ODF: ODF : 4.14.0-99
ACM: 2.9.0-75 (quay.io:443/acm-d/acm-custom-registry:2.9.0-DOWNSTREAM-2023-08-04-03-03-30)
Submariner: 0.16.0 (brew.registry.redhat.io/rh-osbs/iib:549402)
VolSync: 0.7.3
How reproducible:
Always
Steps to Reproduce:
- Deploy 3 OCP 4.14 clusters
- Install RHACM on hub cluster
- Using RHACM console, import other two clusters and connect using Submariner add-ons with globalnet enabled
- Deploy ODF 4.14 on both managed clusters
- Install ODF Multicluster Orchestrator operator on ACM hub cluster
- Create DRPolicy on Hub cluster
- Observe the status of VolSync operator
Actual results:
VolSync operator status Failed and volsync-controller-manager pod in CrashLoopBackOff state.
Expected results:
All pods in running state and Operator status in Succeeded phase.
Additional info:
volsync-controller-manager pod logs from one of the managed cluster:
2023-08-07T05:17:07.483Z INFO setup Go Version: go1.19.9
2023-08-07T05:17:07.483Z INFO setup Go OS/Arch: linux/amd64
2023-08-07T05:17:07.483Z INFO setup Operator Version: ACM-0.7.3-d4eac1c
2023-08-07T05:17:07.483Z INFO setup Rsync container: registry.redhat.io/rhacm2/volsync-rhel8@sha256:5bf810720f7320f01be863a54531e92573240fe07714c24b77b730179bd75c50
2023-08-07T05:17:07.483Z INFO setup Rclone container: registry.redhat.io/rhacm2/volsync-rhel8@sha256:5bf810720f7320f01be863a54531e92573240fe07714c24b77b730179bd75c50
2023-08-07T05:17:07.483Z INFO setup Restic container: registry.redhat.io/rhacm2/volsync-rhel8@sha256:5bf810720f7320f01be863a54531e92573240fe07714c24b77b730179bd75c50
2023-08-07T05:17:07.483Z INFO setup Rsync TLS container: registry.redhat.io/rhacm2/volsync-rhel8@sha256:5bf810720f7320f01be863a54531e92573240fe07714c24b77b730179bd75c50
2023-08-07T05:17:07.483Z INFO setup Syncthing container: registry.redhat.io/rhacm2/volsync-rhel8@sha256:5bf810720f7320f01be863a54531e92573240fe07714c24b77b730179bd75c50
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x138f770]
goroutine 1 [running]:
k8s.io/client-go/discovery.convertAPIResource(...)
/remote-source/volsync/deps/gomod/pkg/mod/k8s.io/client-go@v0.26.1/discovery/aggregated_discovery.go:88
k8s.io/client-go/discovery.convertAPIGroup({{{0x0, 0x0}, {0x0, 0x0}}, {{0xc00005c498, 0x15}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...})
/remote-source/volsync/deps/gomod/pkg/mod/k8s.io/client-go@v0.26.1/discovery/aggregated_discovery.go:69 +0x5f0
k8s.io/client-go/discovery.SplitGroupsAndResources({{{0xc00012a078, 0x15}, {0xc00012c3c0, 0x1b}}, {{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...})
/remote-source/volsync/deps/gomod/pkg/mod/k8s.io/client-go@v0.26.1/discovery/aggregated_discovery.go:35 +0x2f8
k8s.io/client-go/discovery.(*DiscoveryClient).downloadAPIs(0x1ea76c4?)
/remote-source/volsync/deps/gomod/pkg/mod/k8s.io/client-go@v0.26.1/discovery/discovery_client.go:310 +0x47c
k8s.io/client-go/discovery.(*DiscoveryClient).GroupsAndMaybeResources(0xc000369400?)
/remote-source/volsync/deps/gomod/pkg/mod/k8s.io/client-go@v0.26.1/discovery/discovery_client.go:198 +0x5c
k8s.io/client-go/discovery.ServerGroupsAndResources({0x1ed1d48, 0xc000612ba0})
/remote-source/volsync/deps/gomod/pkg/mod/k8s.io/client-go@v0.26.1/discovery/discovery_client.go:392 +0x59
k8s.io/client-go/discovery.(*DiscoveryClient).ServerGroupsAndResources.func1()
/remote-source/volsync/deps/gomod/pkg/mod/k8s.io/client-go@v0.26.1/discovery/discovery_client.go:356 +0x25
k8s.io/client-go/discovery.withRetries(0x2, 0xc0004610a0)
/remote-source/volsync/deps/gomod/pkg/mod/k8s.io/client-go@v0.26.1/discovery/discovery_client.go:621 +0x72
k8s.io/client-go/discovery.(*DiscoveryClient).ServerGroupsAndResources(0x0?)
/remote-source/volsync/deps/gomod/pkg/mod/k8s.io/client-go@v0.26.1/discovery/discovery_client.go:355 +0x3a
k8s.io/client-go/restmapper.GetAPIGroupResources({0x1ed1d48?, 0xc000612ba0?})
/remote-source/volsync/deps/gomod/pkg/mod/k8s.io/client-go@v0.26.1/restmapper/discovery.go:148 +0x42
sigs.k8s.io/controller-runtime/pkg/client/apiutil.NewDynamicRESTMapper.func1()
/remote-source/volsync/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/client/apiutil/dynamicrestmapper.go:94 +0x25
sigs.k8s.io/controller-runtime/pkg/client/apiutil.(*dynamicRESTMapper).setStaticMapper(...)
/remote-source/volsync/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/client/apiutil/dynamicrestmapper.go:130
sigs.k8s.io/controller-runtime/pkg/client/apiutil.NewDynamicRESTMapper(0xc0001242d0?, {0x0, 0x0, 0xa81a664ba2bfe801?})
/remote-source/volsync/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/client/apiutil/dynamicrestmapper.go:110 +0x182
sigs.k8s.io/controller-runtime/pkg/cluster.setOptionsDefaults.func1(0xc0006597a0?)
/remote-source/volsync/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/cluster/cluster.go:217 +0x25
sigs.k8s.io/controller-runtime/pkg/cluster.New(0xc00027cfc0, {0xc000021a48, 0x1, 0x0?})
/remote-source/volsync/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/cluster/cluster.go:159 +0x18d
sigs.k8s.io/controller-runtime/pkg/manager.New(_, {0xc0006597a0, 0x0, 0x0, {{0x1ecde20, 0xc000688480}, 0x0}, 0x1, {0x0, 0x0}, ...})
/remote-source/volsync/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/manager/manager.go:359 +0xf9
main.main()
/remote-source/volsync/app/main.go:189 +0x25a
- links to
-
RHEA-2023:118321
VolSync 0.7 for RHEL 8
