This RFE proposes enhancing the ansible.eda.kafka source plugin within Red Hat Ansible Automation Platform (EDA) to allow the topic parameter to accept wildcards or regular expressions for subscribing to Kafka topics. Currently, the topic parameter only supports exact string matches for topic names.

The ability to use patterns (like gh-status.* or cluster-events-.+) would enable a single Kafka source definition to dynamically subscribe to multiple topics that match a given pattern, without explicitly listing each topic. This is crucial for environments where topic names are generated dynamically or follow a predictable naming convention.

Business Requirements

1. Automated Topic Inclusion in Dynamic Environments: In highly dynamic environments, such as those integrated with Red Hat Advanced Cluster Management for Kubernetes (RHACM) and MultiClusterGlobalHub, new Kafka topics are frequently created automatically (e.g., when new OpenShift clusters are provisioned, each might get a dedicated Kafka topic). Without wildcard support, every time a new topic is created, the rulebook activation must be stopped, the topic variable (or a new topics list) updated, and the rulebook activation restarted. This manual process is inefficient, prone to human error, and introduces downtime for the automation.

1. Scalability and Reduced Operational Overhead: The manual management of topic lists for hundreds or thousands of clusters is not scalable. Wildcard support significantly reduces operational overhead by allowing a "set it and forget it" approach for topics following a naming convention, improving automation efficiency.

1. Simplified Rulebook Management: Rulebooks can be designed to be more generic and resilient to changes in topic names, leading to simpler maintenance and easier deployment across different environments.

1. Enabling True Event-Driven Automation: To fully leverage event-driven automation in dynamic cloud-native setups, EDA needs the capability to automatically react to events from new, unforeseen, but predictably named Kafka topics without manual configuration changes.

Functional Requirements

Modify the ansible.eda.kafka source plugin: The plugin should be updated to interpret the value of the topic parameter (or a new topic_pattern parameter) as a wildcard or regular expression, rather than a literal string.

• Example Proposed Syntax:

- name: Read messages from Kafka topics matching a pattern
  hosts: all
  sources:
    - ansible.eda.kafka:
        host: "{{ HOST }}"
        port: 443
        topic: "gh-status.*" # Subscribe to all topics starting with "gh-status."
        # Alternatively, if a new parameter is preferred:
        # topic_pattern: "gh-status-cluster-[a-z0-9]+"
        check_hostname: false
        cafile: "{{ eda.filename.ca_cert }}"
        certfile: "{{ eda.filename.client_cert }}"
        keyfile: "{{ eda.filename.client_key }}"
        verify_mode: CERT_NONE
  rules:
    - name: Process Dynamic Cluster Events
      # ... rule conditions and actions

Dependencies

• Documentation: The official Red Hat Ansible Automation Platform documentation, specifically the Ansible Rulebook guide and the ansible.eda.kafka source plugin documentation, would need to be updated to detail the usage of wildcards/regular expressions for topic subscriptions.

• Ansible Automation Platform UI / Event-Driven Ansible Controller:

• • Any UI elements related to creating or editing rulebooks that involve Kafka sources might need adjustments to clarify that the topic field can now accept patterns.

• • The schema validation for rulebook definitions within the controller would need to be updated to accommodate pattern strings.

• ansible.eda.kafka Collection: The collection itself would require code changes to implement this pattern matching and subscription logic.