• Icon: Bug Bug
    • Resolution: Done
    • Icon: Undefined Undefined
    • rhosdt-3.6
    • None
    • Tempo
    • None
    • Tracing Sprint # 266, Tracing Sprint # 268 - release, Tracing Sprint # 269

      Sometimes, the tracing UI shows this error:

      JSON.parse: unexpected character at line 1 column 1 of the JSON data
      

      The web dev tools show that the gateway redirects to the oauth login page.

            [TRACING-5098] Fix token expiration in Tracing UI plugin

            Improve error message on Perses side: https://github.com/perses/plugins/pull/93

            Andreas Gerstmayr added a comment - Improve error message on Perses side: https://github.com/perses/plugins/pull/93

            The problem is that the naming of the cluster-wide ClusterRole and ClusterRoleBinding: we use "tempo-<instance>-gateway", and if we create a Tempo instance called "sample" in namespace A and another instance also called "sample" in namespace B, the ClusterRoleBinding gets overwritten because it's always "tempo-sample-gateway".

            Andreas Gerstmayr added a comment - The problem is that the naming of the cluster-wide ClusterRole and ClusterRoleBinding: we use "tempo-<instance>-gateway", and if we create a Tempo instance called "sample" in namespace A and another instance also called "sample" in namespace B, the ClusterRoleBinding gets overwritten because it's always "tempo-sample-gateway".

            Afaics this is related to this message on the gateway:

            E0227 14:46:22.646025 1 webhook.go:154] Failed to make webhook authenticator request: tokenreviews.authentication.k8s.io is forbidden: User "system:serviceaccount:edge-monitoring:tempo-sample-gateway" cannot create resource "tokenreviews" in API group "authentication.k8s.io" at the cluster scope
            level=info name=observatorium ts=2025-02-27T14:46:22.646051154Z caller=openshift.go:436 msg="fallback to read cookie, no serviceaccount bearer token or mTLS certs provided"
            

            The token is likely correct, but the gateway fails to verify it because it doesn't have permissions to create the tokenreview, therefore it redirects to the login page.

            Andreas Gerstmayr added a comment - Afaics this is related to this message on the gateway: E0227 14:46:22.646025 1 webhook.go:154] Failed to make webhook authenticator request: tokenreviews.authentication.k8s.io is forbidden: User "system:serviceaccount:edge-monitoring:tempo-sample-gateway" cannot create resource "tokenreviews" in API group "authentication.k8s.io" at the cluster scope level=info name=observatorium ts=2025-02-27T14:46:22.646051154Z caller=openshift.go:436 msg="fallback to read cookie, no serviceaccount bearer token or mTLS certs provided" The token is likely correct, but the gateway fails to verify it because it doesn't have permissions to create the tokenreview, therefore it redirects to the login page.

              agerstma@redhat.com Andreas Gerstmayr
              agerstma@redhat.com Andreas Gerstmayr
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: