-
Story
-
Resolution: Obsolete
-
Minor
-
None
-
Logging 5.3.0
-
False
-
False
-
NEW
-
NEW
-
Story
As an administrator of Cluster LogForwarding,
I want configure additional KAFKA SASL options
so I can forward logs using my configured authentication mechanism
Acceptance Criteria
Notes
When set sasl.enable,sasl.mechanisms,sasl.allow-insecure and passphrase in secret. the relevant options can't be created in fluent.conf
Step to reproduce:
1. create fluentd_to_kafka secret
oc create secret generic kafka-fluentd --from-file=ca-bundle.crt=ca/ca_bundle.crt --from-file=tls.crt=client/client.crt --from-file=tls.key=client/client.key --from-literal=username=${kafka_user_name} --from-literal=password=${kafka_user_password} --from-literal=sasl.enable=false --from-literal=sasl.mechanisms=GSSAPI,PLAIN --from-literal=passphrase=aosqe2021 -n openshift-logging
2. create clusterloggingforwarder
"spec": { "outputs": [ { "name": "kafka-app", "secret": { "name": "kafka-fluentd" }, "type": "kafka", "url": "tls://kafka.openshift-logging.svc.cluster.local:9093/clo-topic" } ], "pipelines": [ { "inputRefs": [ "application" ], "name": "test-app", "outputRefs": [ "kafka-app" ] } ] },
Actual result:
# Ship logs to specific outputs
<label @KAFKA_APP>
<match **>
@type kafka2
@id kafka_app
brokers kafka.openshift-logging.svc.cluster.local:9093
default_topic clo-topic
use_event_time true
username "#{File.exists?('/var/run/ocp-collector/secrets/kafka-fluentd/username') ? open('/var/run/ocp-collector/secrets/kafka-fluentd/username','r') do |f|f.read end : ''}"
password "#{File.exists?('/var/run/ocp-collector/secrets/kafka-fluentd/password') ? open('/var/run/ocp-collector/secrets/kafka-fluentd/password','r') do |f|f.read end : ''}"
ssl_client_cert_key '/var/run/ocp-collector/secrets/kafka-fluentd/tls.key'
ssl_client_cert '/var/run/ocp-collector/secrets/kafka-fluentd/tls.crt'
ssl_ca_cert '/var/run/ocp-collector/secrets/kafka-fluentd/ca-bundle.crt'
sasl_over_ssl true
<format>
@type json
</format>
<buffer clo-topic>
....
</buffer>
</match>
Expected result
# Ship logs to specific outputs
<label @KAFKA_APP>
<match **>
@type kafka2
@id kafka_app
brokers kafka.openshift-logging.svc.cluster.local:9093
default_topic clo-topic
use_event_time true
username "#{File.exists?('/var/run/ocp-collector/secrets/kafka-fluentd/username') ? open('/var/run/ocp-collector/secrets/kafka-fluentd/username','r') do |f|f.read end : ''}"
password "#{File.exists?('/var/run/ocp-collector/secrets/kafka-fluentd/password') ? open('/var/run/ocp-collector/secrets/kafka-fluentd/password','r') do |f|f.read end : ''}"
ssl_client_cert_key '/var/run/ocp-collector/secrets/kafka-fluentd/tls.key'
ssl_client_cert '/var/run/ocp-collector/secrets/kafka-fluentd/tls.crt'
ssl_ca_cert '/var/run/ocp-collector/secrets/kafka-fluentd/ca-bundle.crt'
sasl_over_ssl false
passphrase aosqe2021
sasl_scram_mechanism: 'sha256'
@type json
</format>
<buffer clo-topic>
....
</buffer>
</match>
[LOG-1876] Untangle CLF Kafka SASL configuration options
Added needs-discussion and de-prioritizing given we have no customer asks at this time for these auth mechanisms
Converting this to an enhancement given the following slack conversation
@aconway there is this bug against the auth expansion for fluent https://issues.redhat.com/browse/LOG-1876 which we are missing these keys. My investigation leads me to believe we need to provide a priority of some kind or some logic to fill out combinations. It looks like there are say multiple ways to specify "username" depending on what other things you enable. same with password. The options seem out of control and it seems like there would have been a better way to combine similar concept fields. I'm not sure why 'sasl_username' needs to be a different config option then 'username'. Can you read the issue and commment
Jeff Cantrill 1 day ago
One way to address is to modify the api docs and make these changes an enhancement instead of a bug :man-shrugging:Alan Conway:spiral_calendar_pad: 21 hours ago
Will take a look. We are trying to present a simpler configuration model than fluentd does, so we are probably missing some logic to match the two. Fluentd takes the approach that every security mech is it's own thing and doesn't try to line up concepts ilke username in a uniform way. I think I can propose a solution...Anping Li 18 hours ago
@jcantril @aconway I prefer a enhancement. I think there are some new logic to use these options.Alan Conway:spiral_calendar_pad: 3 hours ago
That's fine with me. It is an enhancement really, we never claimed the config would work like this before.Jeff Cantrill 3 hours ago
@aconway I believe the only variance and the source of @anli’s bug is we document this info in the api doc
Closing this obsolete pending the release of logging 6.0 which added explicit auth configuration and SASL options. We may need to provide additional enhancement but we'll wait for specific customer asks and write a new issue