Uploaded image for project: 'Red Hat Advanced Cluster Management'
  1. Red Hat Advanced Cluster Management
  2. ACM-11494

Cluster proxy pods crashlooping in mce 2.6

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Normal Normal
    • None
    • MCE 2.6.0
    • Cluster Lifecycle
    • None
    • None
    • False
    • No
    • Important

      On fresh install of latest downstream of acm 2.11

      Cluster proxy pods seem to be crashing in acm 2.11/mce 2.6

      oc get pods | grep "cluster-proxy"
      cluster-proxy-6db985ff9b-2bsq8                         0/1     CrashLoopBackOff   236 (2m33s ago)   19h
      cluster-proxy-6db985ff9b-b5qsn                         0/1     CrashLoopBackOff   236 (3m59s ago)   19h
      cluster-proxy-addon-manager-6d9c9c5cb9-5fhk7           1/1     Running            0                 19h
      cluster-proxy-addon-manager-6d9c9c5cb9-zkn9n           1/1     Running            0                 19h
      cluster-proxy-addon-user-774bf9c5b6-2s5j2              2/2     Running            0                 19h
      cluster-proxy-addon-user-774bf9c5b6-fx89z              2/2     Running            0                 19h

      Logs

      oc logs cluster-proxy-6db985ff9b-2bsq8
      I0507 13:01:10.776041       1 options.go:148] ServerCert set to "/etc/server-pki/tls.crt".
      I0507 13:01:10.776115       1 options.go:149] ServerKey set to "/etc/server-pki/tls.key".
      I0507 13:01:10.776120       1 options.go:150] ServerCACert set to "/etc/server-ca-pki/ca.crt".
      I0507 13:01:10.776123       1 options.go:151] ClusterCert set to "/etc/agent-pki/tls.crt".
      I0507 13:01:10.776127       1 options.go:152] ClusterKey set to "/etc/agent-pki/tls.key".
      I0507 13:01:10.776131       1 options.go:153] ClusterCACert set to "/etc/server-ca-pki/ca.crt".
      I0507 13:01:10.776136       1 options.go:154] Mode set to "grpc".
      I0507 13:01:10.776140       1 options.go:155] UDSName set to "".
      I0507 13:01:10.776144       1 options.go:156] DeleteUDSFile set to true.
      I0507 13:01:10.776149       1 options.go:157] Server port set to 8090.
      I0507 13:01:10.776154       1 options.go:158] Server bind address set to "".
      I0507 13:01:10.776159       1 options.go:159] Agent port set to 8091.
      I0507 13:01:10.776164       1 options.go:160] Agent bind address set to "".
      I0507 13:01:10.776170       1 options.go:161] Admin port set to 8095.
      I0507 13:01:10.776175       1 options.go:162] Admin bind address set to "127.0.0.1".
      I0507 13:01:10.776181       1 options.go:163] Health port set to 8092.
      I0507 13:01:10.776187       1 options.go:164] Health bind address set to "".
      I0507 13:01:10.776193       1 options.go:165] Keepalive time set to 30s.
      I0507 13:01:10.776206       1 options.go:166] Frontend keepalive time set to 1h0m0s.
      I0507 13:01:10.776213       1 options.go:167] EnableProfiling set to false.
      I0507 13:01:10.776220       1 options.go:168] EnableContentionProfiling set to false.
      I0507 13:01:10.776227       1 options.go:169] ServerID set to a1335309-085a-418a-91ae-8ea65d07c66a.
      I0507 13:01:10.776234       1 options.go:170] ServerCount set to 2.
      I0507 13:01:10.776241       1 options.go:171] AgentNamespace set to "".
      I0507 13:01:10.776249       1 options.go:172] AgentServiceAccount set to "".
      I0507 13:01:10.776257       1 options.go:173] AuthenticationAudience set to "".
      I0507 13:01:10.776264       1 options.go:174] KubeconfigPath set to "".
      I0507 13:01:10.776272       1 options.go:175] KubeconfigQPS set to 0.000000.
      I0507 13:01:10.776285       1 options.go:176] KubeconfigBurst set to 0.
      I0507 13:01:10.776294       1 options.go:177] ProxyStrategies set to "destHost".
      I0507 13:01:10.776303       1 options.go:178] CipherSuites set to ["TLS_AES_256_GCM_SHA384" "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA" "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256" "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384" "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256" "TLS_RSA_WITH_AES_256_CBC_SHA" "TLS_RSA_WITH_AES_128_GCM_SHA256" "TLS_AES_128_GCM_SHA256" "TLS_CHACHA20_POLY1305_SHA256" "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA" "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256" "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" "TLS_RSA_WITH_AES_128_CBC_SHA" "TLS_RSA_WITH_AES_256_GCM_SHA384" "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384" "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA" "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA"].
      Error: failed to validate server options with cipher suite TLS_RSA_WITH_AES_256_CBC_SHA not supported, doesn't exist or considered as insecure
      Usage:
        proxy [flags]

      Flags:
            --add-dir-header                     If true, adds the file directory to the header of the log messages
            --admin-bind-address string          Bind address for admin connections. If empty, we will bind to localhost. (default "127.0.0.1")
            --admin-port int                     Port we listen for admin connections on. (default 8095)
            --agent-bind-address string          Bind address for agent connections. If empty, we will bind to all interfaces.
            --agent-namespace string             Expected agent's namespace during agent authentication (used with agent-service-account, authentication-audience, kubeconfig).
            --agent-port int                     Port we listen for agent connections on. (default 8091)
            --agent-service-account string       Expected agent's service account during agent authentication (used with agent-namespace, authentication-audience, kubeconfig).
            --alsologtostderr                    log to standard error as well as files (no effect when -logtostderr=true)
            --authentication-audience string     Expected agent's token authentication audience (used with agent-namespace, agent-service-account, kubeconfig).
            --cipher-suites strings              The comma separated list of allowed cipher suites. Has no effect on TLS1.3. Empty means allow default list.
            --cluster-ca-cert string             If non-empty the CA we use to validate Agent clients.
            --cluster-cert string                If non-empty secure communication with this cert.
            --cluster-key string                 If non-empty secure communication with this key.
            --delete-existing-uds-file           If true and if file UdsName already exists, delete the file before listen on that UDS file. Default is true. (default true)
            -enable-contention-profiling        enable contention profiling at host:admin-port/debug/pprof/block. "-enable-profiling" must also be set.
            --enable-profiling                   enable pprof at host:admin-port/debug/pprof
            --frontend-keepalive-time duration   Time for gRPC frontend server keepalive. (default 1h0m0s)
            --health-bind-address string         Bind address for health connections. If empty, we will bind to all interfaces.
            --health-port int                    Port we listen for health connections on. (default 8092)
        -h, --help                               help for proxy
            --keepalive-time duration            Time for gRPC agent server keepalive. (default 1h0m0s)
            --kubeconfig string                  absolute path to the kubeconfig file (used with agent-namespace, agent-service-account, authentication-audience).
            --kubeconfig-burst int               Maximum client burst (proxy server uses this client to authenticate agent tokens).
            --kubeconfig-qps float32             Maximum client QPS (proxy server uses this client to authenticate agent tokens).
            --log-backtrace-at traceLocation     when logging hits line file:N, emit a stack trace (default :0)
            --log-dir string                     If non-empty, write log files in this directory (no effect when -logtostderr=true)
            --log-file string                    If non-empty, use this log file (no effect when -logtostderr=true)
            --log-file-max-size uint             Defines the maximum size a log file can grow to (no effect when -logtostderr=true). Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
            --logtostderr                        log to standard error instead of files (default true)
            --mode string                        mode can be either 'grpc' or 'http-connect'. (default "grpc")
            --one-output                         If true, only write logs to their native severity level (vs also writing to each lower severity level; no effect when -logtostderr=true)
            --proxy-strategies string            The list of proxy strategies used by the server to pick a backend/tunnel, available strategies are: default, destHost. (default "default")
            --server-bind-address string         Bind address for server connections. If empty, we will bind to all interfaces.
            --server-ca-cert string              If non-empty the CA we use to validate KAS clients.
            --server-cert string                 If non-empty secure communication with this cert.
            --server-count uint                  The number of proxy server instances, should be 1 unless it is an HA server. (default 1)
            --server-id string                   The unique ID of this server. Can also be set by the 'PROXY_SERVER_ID' environment variable. (default "a1335309-085a-418a-91ae-8ea65d07c66a")
            --server-key string                  If non-empty secure communication with this key.
            --server-port int                    Port we listen for server connections on. Set to 0 for UDS. (default 8090)
            --skip-headers                       If true, avoid header prefixes in the log messages
            --skip-log-headers                   If true, avoid headers when opening log files (no effect when -logtostderr=true)
            --stderrthreshold severity           logs at or above this threshold go to stderr when writing to files and stderr (no effect when -logtostderr=true or -alsologtostderr=false) (default 2)
            --uds-name string                    uds-name should be empty for TCP traffic. For UDS set to its name.
        -v, --v Level                            number for the log level verbosity (default 4)
            --vmodule moduleSpec                 comma-separated list of pattern=N settings for file-filtered logging

      E0507 13:01:10.776694       1 main.go:49] error: failed to validate server options with cipher suite TLS_RSA_WITH_AES_256_CBC_SHA not supported, doesn't exist or considered as insecure

            leyan@redhat.com Le Yang
            kurwang@redhat.com Kurtis Wang
            David Huynh David Huynh
            ACM QE Team
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: