Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-8074

telemetry-operator panics when using customMonitoringStack and ScrapeInterval isn't set

XMLWordPrintable

    • 1
    • False
    • Hide

      None

      Show
      None
    • False
    • ?
    • ?
    • ?
    • ?
    • CloudOps 2024 Sprint 14
    • Moderate

      • related compose
        • quay.io/openstack-k8s-operators/telemetry-operator@sha256:ed7cce33b041dac161561a718f27cdb69f1e9fb561c71ec635885ffa3f1c1707
      • _How reproducible
        _
        • Always
      • _Steps to reproduce
        _
        • Use the customMonitoringStack field, but don't set the prometheusConfig.scrapeInterval
      • _Expected result
        _
        • MetricStorage works with some default value or user is informed about what's wrong. Not a panic.
      • _Actual results
        _
        • telemetry-operator panics without any info for the user
      • Additional info:
        • Used MonitoringStack
        •   spec:
              alertmanagerConfig:
                disabled: false
              logLevel: debug
              prometheusConfig:
                replicas: 1
              resourceSelector:
                matchLabels:
                  service: ceilometer
              resources:
                limits:
                  cpu: 500m
                  memory: 512Mi
                requests:
                  cpu: 100m
                  memory: 256Mi
              retention: 1d 
        • telemetry-operator panic log traceback
        • 2024-06-27T17:31:30Z    INFO    Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference    {"controller": "metricstorage", "controllerGroup": "telemetry.openstack.org", "controllerKind": "MetricStorage", "MetricStorage": {"name":"metric-storage","namespace":"openstack"}, "namespace": "openstack", "name": "metric-storage", "reconcileID": "1dcf74b9-6bde-4b21-88d3-1a9b1e1da860"}
          panic: runtime error: invalid memory address or nil pointer dereference [recovered]
              panic: runtime error: invalid memory address or nil pointer dereference
          [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x189a5d3]goroutine 536 [running]:
          sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
              /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.6/pkg/internal/controller/controller.go:116 +0x1fa
          panic({0x1acb660, 0x2db1a70})
              /usr/lib/golang/src/runtime/panic.go:884 +0x213
          github.com/openstack-k8s-operators/telemetry-operator/pkg/metricstorage.ServiceMonitor(0x1aaeba0?, 0xc000411050, 0xc0018036e0, {0xc0070c92c0, 0x21}, {0x0, 0x0})
              /remote-source/pkg/metricstorage/service_monitor.go:39 +0x93
          github.com/openstack-k8s-operators/telemetry-operator/controllers.(*MetricStorageReconciler).reconcileNormal.func3()
              /remote-source/controllers/metricstorage_controller.go:393 +0xc5
          sigs.k8s.io/controller-runtime/pkg/controller/controllerutil.mutate(0x2008d00?, {{0xc000696140?, 0x0?}, {0xc0070c92f0?, 0x2020f20?}}, {0x2038ea8, 0xc0071721e0})
              /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.6/pkg/controller/controllerutil/controllerutil.go:340 +0x4f
          sigs.k8s.io/controller-runtime/pkg/controller/controllerutil.CreateOrPatch({0x2020f20, 0xc002aa0570}, {0x202b580, 0xc000574630}, {0x2038ea8?, 0xc0071721e0}, 0xc005df5d50)
              /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.6/pkg/controller/controllerutil/controllerutil.go:240 +0x150
          github.com/openstack-k8s-operators/telemetry-operator/controllers.(*MetricStorageReconciler).reconcileNormal(0xc0002a1570, {0x2020f20, 0xc002aa0570}, 0xc002a89760, 0xc000654630)
              /remote-source/controllers/metricstorage_controller.go:389 +0x162a
          github.com/openstack-k8s-operators/telemetry-operator/controllers.(*MetricStorageReconciler).Reconcile(0xc0002a1570, {0x2020f20, 0xc002aa0570}, {{{0xc000696140?, 0x5?}, {0xc000696130?, 0xc000f23d08?}}})
              /remote-source/controllers/metricstorage_controller.go:200 +0xfe5
          sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x20236a0?, {0x2020f20?, 0xc002aa0570?}, {{{0xc000696140?, 0xb?}, {0xc000696130?, 0x0?}}})
              /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.6/pkg/internal/controller/controller.go:119 +0xc8
          sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0000fe780, {0x2020e78, 0xc0003a7950}, {0x1b85460?, 0xc000c92000?})
              /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.6/pkg/internal/controller/controller.go:316 +0x3ca
          sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0000fe780, {0x2020e78, 0xc0003a7950})
              /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.6/pkg/internal/controller/controller.go:266 +0x1c5
          sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
              /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.6/pkg/internal/controller/controller.go:227 +0x85
          created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
              /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.6/pkg/internal/controller/controller.go:223 +0x587 

            rh-ee-jwysogla Jaromir Wysoglad
            rh-ee-jwysogla Jaromir Wysoglad
            rhos-dfg-cloudops
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: