Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-27068

Ceilometer Compute TLS does not get correctly configured

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Normal Normal
    • rhos-18.0.18
    • rhos-18.0.17 FR 5
    • telemetry-operator
    • None
    • 0
    • False
    • Hide

      None

      Show
      None
    • False
    • ?
    • rhos-conplat-observability
    • None
    • Hide
      Metrics not being scraped from the Ceilometer compute agent on data plane nodes::
      In RHOSO 18.0.17, the Ceilometer compute agent, `ceilometer_agent_compute`, fails to start because the data plane does not provision TLS certificates for the Ceilometer `prometheus_exporter`. This results in Prometheus being unable to scrape metrics from the Ceilometer compute agent on data plane nodes.
      +
      *Workaround:* Configure the missing property as `customServiceConfig` in the `OpenStackControlPlane` CR:
      +
      ----
      apiVersion: core.openstack.org/v1beta1
      kind: OpenStackControlPlane
      metadata:
        name: openstack-control-plane
      spec:
        telemetry:
          template:
            ceilometer:
              customServiceConfig: |
                [service_credentials]
                cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
      ----
      Show
      Metrics not being scraped from the Ceilometer compute agent on data plane nodes:: In RHOSO 18.0.17, the Ceilometer compute agent, `ceilometer_agent_compute`, fails to start because the data plane does not provision TLS certificates for the Ceilometer `prometheus_exporter`. This results in Prometheus being unable to scrape metrics from the Ceilometer compute agent on data plane nodes. + *Workaround:* Configure the missing property as `customServiceConfig` in the `OpenStackControlPlane` CR: + ---- apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane metadata:   name: openstack-control-plane spec:   telemetry:     template:       ceilometer:         customServiceConfig: |           [service_credentials]           cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem ----
    • Known Issue
    • Bug Delivery Tracker
    • 1
    • Important

      Goal

      When telemetry is enabled with metricStorage on an RHOSO 18 deployment, the ceilometer compute agent (ceilometer_agent_compute) deployed on EDPM nodes fails to start due to missing TLS configuration. The data plane telemetry service sets prometheus_tls_enable = True and references /etc/ceilometer/tls/tls.crt and /etc/ceilometer/tls/tls.key in the generated ceilometer.conf, but neither the TLS certificates nor the required volume mounts are provisioned during the data plane deployment. Additionally, the [service_credentials] section lacks a cafile setting, preventing ceilometer from verifying Keystone's SSL certificate.

      The telemetry data plane Ansible roles should provision TLS certificates for the ceilometer prometheus exporter, mount them into the container, and configure the CA trust chain for Keystone authentication.

      Problem details

      Issue 1 — Missing TLS cert/key for prometheus exporter:

      The container crashes on startup with:

      FileNotFoundError: Cannot load server certificate file '/etc/ceilometer/tls/tls.crt'
or its private key file '/etc/ceilometer/tls/tls.key': [Errno 2] No such file or directory

      The config references these files but nothing generates or deploys them. The container also lacks a volume mount for /etc/ceilometer/tls/.

      Issue 2 — Keystone SSL verification failure (missing CA bundle):

      After fixing Issue 1, ceilometer cannot authenticate to Keystone:

      SSLCertVerificationError: certificate verify failed: unable to get local issuer certificate

      The [service_credentials] section of ceilometer.conf has no cafile set. The container bind-mounts /etc/pki/ca-trust/source/anchors from the host (which contains the OpenStack internal root CA), but the ceilometer user inside the container cannot read that directory due to permissions.

      Acceptance Criteria

      • Verify that when telemetry with metricStorage is enabled, TLS certificates signed by rootca-internal are generated and deployed to /etc/ceilometer/tls/ on each EDPM compute node during the data plane deployment
      • Verify that the ceilometer_agent_compute container is created with a volume mount for /etc/ceilometer/tls so the certificates are accessible inside the container
      • Verify that the generated ceilometer.conf includes cafile in the [service_credentials] section pointing to a CA bundle that contains the rootca-internal certificate, at a path readable by the ceilometer user inside the container
      • Verify that ceilometer_agent_compute starts without TLS-related errors and the prometheus exporter listens on port 9101 with TLS
      • Verify that ceilometer can authenticate to Keystone and successfully discover/poll compute instances
      • Alternatively, if TLS for the prometheus exporter is not required on compute nodes, verify that the config sets prometheus_tls_enable = False so certificates are not needed

              rhn-engineering-jlarriba Juan Larriba
              rhn-engineering-jlarriba Juan Larriba
              rhos-observability-telemetry
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: