Uploaded image for project: 'AI Platform Core Components'
  1. AI Platform Core Components
  2. AIPCC-10742

Add apache-airflow into the RHAI pipeline onboarding collection

    • Icon: Story Story
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • None
    • None
    • False
    • Hide

      None

      Show
      None
    • False

      Add package 'apache-airflow' into the RHAI pipeline onboarding collection.

      The package requires builder repository onboarding before it can be added to the RHAI pipeline. This ticket is blocked by the builder onboarding ticket.

      Summary

      Here is the executive summary formatted as a JIRA comment using JIRA wiki markup:

      Executive Summary: apache-airflow Packaging Analysis

      apache-airflow (v3.1.7) is a pure Python package with a build complexity score of 0/10, meaning it requires no native compilation (C/C++/Rust/Fortran) and ships as a universal wheel (

      none-any

      ). Building a wheel from the source distribution is straightforward and does not require specialized build tooling beyond a standard Python environment. The package requires Python >=3.10, <3.14. From a pure build perspective, this package presents no significant blockers for onboarding.

      The primary complexity of apache-airflow lies not in compilation but in its massive dependency ecosystem. The package declares 237 dependencies, the vast majority of which are optional provider packages gated behind extras (e.g.,

      apache-airflow-providers-amazon

      ,

      apache-airflow-providers-google

      ,

      apache-airflow-providers-cncf-kubernetes

      ). The two hard runtime dependencies are

      apache-airflow-core==3.1.7

      and

      apache-airflow-task-sdk==1.1.7

      . Several of the optional provider dependencies pull in packages that do require compilation (e.g.,

      python-ldap

      , database drivers, gRPC libraries), so the transitive dependency surface must be carefully scoped based on which extras are enabled. Some provider packages also carry Python version restrictions (e.g.,

      apache-beam

      ,

      fab

      ,

      yandex

      , and

      ydb

      exclude Python 3.13).

      Key recommendations for onboarding:

      • Building the core
        apache-airflow

        wheel itself is trivial — source distribution is available and the wheel is architecture-independent.

      • Define the required extras early — the dependency footprint varies dramatically between a minimal install (
        apache-airflow-core

        +

        task-sdk

        ) and a full install with all 100+ providers. Each selected extra may introduce its own transitive compilation requirements.

      • Investigate
        apache-airflow-core

        separately, as it is the actual runtime package and may carry additional build considerations not visible at this meta-package level.

      • Provider packages that pull in compiled dependencies (e.g.,
        ldap

        ,

        grpc

        ,

        odbc

        ,

        mysql

        ,

        postgres

        ) should be individually assessed for platform-specific wheel availability and build toolchain requirements.

              epacific@redhat.com Einat Pacifici
              aipcc-jira-bot@redhat.com AIPCC JIRABOT
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: