-
Bug
-
Resolution: Done
-
Undefined
-
None
-
None
-
1
-
False
-
None
-
False
-
-
-
Tracing Sprint # 257
Version of components:
OCP version: 4.16.0-0.nightly-2024-07-17-183709
tempo-operator.v0.11.1
Description of the problem:
When a Tempo Monolithic instance is created with TLS enabled and without setting any custom certs, the instance fails to create with error ''
MountVolume.SetUp failed for volume "tempo-mono-cert-serving-cert" : secret "tempo-mono-cert-serving-cert" not found % oc get pods NAME READY STATUS RESTARTS AGE minio-744f658d8d-c5rxq 1/1 Running 0 27s tempo-mono-cert-0 0/2 ContainerCreating 0 7s % oc describe pod tempo-mono-cert-0 Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 17s default-scheduler Successfully assigned chainsaw-tls-mono-st/tempo-mono-cert-0 to ip-10-0-13-81.us-east-2.compute.internal Warning FailedMount 2s (x6 over 17s) kubelet MountVolume.SetUp failed for volume "tempo-mono-cert-serving-cert" : secret "tempo-mono-cert-serving-cert" not found % oc get cm NAME DATA AGE kube-root-ca.crt 1 15m openshift-service-ca.crt 1 15m tempo-mono-cert-config 2 15m tempo-mono-cert-serving-cabundle 1 15m % oc get secrets NAME TYPE DATA AGE builder-dockercfg-vlwb4 kubernetes.io/dockercfg 1 15m default-dockercfg-tdw4f kubernetes.io/dockercfg 1 15m deployer-dockercfg-vrgkv kubernetes.io/dockercfg 1 15m minio Opaque 4 15m tempo-mono-cert-dockercfg-pbr9f kubernetes.io/dockercfg 1 15m
Steps to reproduce the issue:
1. Install the Tempo operator built off the latest upstream branch.
2. Run the following steps to create the monolithic instance.
oc new-project chainsaw-tls-mono-st cat 00-install-storage.yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: labels: app.kubernetes.io/name: minio name: minio namespace: chainsaw-tls-mono-st spec: accessModes: - ReadWriteOnce resources: requests: storage: 2Gi --- apiVersion: apps/v1 kind: Deployment metadata: name: minio namespace: chainsaw-tls-mono-st spec: selector: matchLabels: app.kubernetes.io/name: minio strategy: type: Recreate template: metadata: labels: app.kubernetes.io/name: minio spec: containers: - command: - /bin/sh - -c - | mkdir -p /storage/tempo && \ minio server /storage env: - name: MINIO_ACCESS_KEY value: tempo - name: MINIO_SECRET_KEY value: supersecret image: minio/minio name: minio ports: - containerPort: 9000 volumeMounts: - mountPath: /storage name: storage volumes: - name: storage persistentVolumeClaim: claimName: minio --- apiVersion: v1 kind: Service metadata: name: minio namespace: chainsaw-tls-mono-st spec: ports: - port: 9000 protocol: TCP targetPort: 9000 selector: app.kubernetes.io/name: minio type: ClusterIP --- apiVersion: v1 kind: Secret metadata: name: minio namespace: chainsaw-tls-mono-st stringData: endpoint: http://minio:9000 bucket: tempo access_key_id: tempo access_key_secret: supersecret type: Opaque oc create -f 00-install-storage.yaml cat 01-install-tempo.yaml apiVersion: tempo.grafana.com/v1alpha1 kind: TempoMonolithic metadata: name: mono-cert namespace: chainsaw-tls-mono-st spec: jaegerui: enabled: true ingestion: otlp: grpc: tls: enabled: true http: tls: enabled: true oc create -f 01-install-tempo.yaml
3. Check the monolithic pod status. Its stuck in ContainerCreating state with the mount error in the pod events.