-
Story
-
Resolution: Unresolved
-
Undefined
-
None
-
None
-
None
-
False
-
-
False
-
-
Add package 'hnswlib' into the RHAI pipeline onboarding collection.
The package requires builder repository onboarding before it can be added to the RHAI pipeline. This ticket is blocked by the builder onboarding ticket.
Summary
Here is the executive summary formatted as a JIRA comment in JIRA wiki markup:
Executive Summary: hnswlib Build Analysis
hnswlib (v0.8.0) is a C++ header-only approximate nearest neighbor search library with Python bindings via pybind11. It requires source compilation — no pre-built wheels exist on PyPI for any platform or version. The build itself is moderate complexity (4/10): a single C++ source file (
python_bindings/bindings.cpp
) compiled against pybind11 and numpy headers, with no external C/C++ library dependencies. The package is licensed under Apache-2.0, which is fully compatible with Red Hat redistribution requirements.
Three critical blockers must be resolved before building for enterprise distribution. First, the
PYBIND11_PLUGIN
macro used in
bindings.cpp:910
was removed in pybind11 3.x, and the pyproject.toml has no upper bound on pybind11 — the build will fail unless pybind11 is pinned to
<3.0
or the source is patched to use
PYBIND11_MODULE
. Second,
setup.py:57
references
setuptools.distutils.errors.CompileError
, an import path broken on Python 3.12+ — this must be patched. Third, the default
-march=native
compiler flag produces non-portable binaries that crash with "Illegal instruction" on CPUs different from the build host; the environment variable
HNSWLIB_NO_NATIVE=1
must be set for all distribution builds.
The recommended build approach is:
export HNSWLIB_NO_NATIVE=1 # Apply patches for pybind11 <3.0 pin and distutils import fix pip wheel . --no-build-isolation
For x86_64, the library provides runtime SIMD dispatch (SSE/AVX/AVX-512) via CPUID checks, so portable binaries still achieve optimal performance on capable hardware. For aarch64, no SIMD optimizations exist — the library falls back to scalar code with significantly lower performance. The upstream project is effectively unmaintained: 271 open issues, last release December 2023, and CI only tests up to Python 3.10.
Key stakeholder note: hnswlib is a CPU-only library with no CUDA or ROCm support. If the onboarding requirement (per AIPCC-10799) includes GPU-accelerated vector search indexes, hnswlib cannot satisfy that requirement. Alternatives such as faiss-gpu or NVIDIA cuVS/RAFT should be evaluated for GPU index targets. hnswlib can only serve the CPU index use case.
- blocks
-
AIPCC-10816 Onboard hnswlib into the AIPCC Builder
-
- In Progress
-
- mentioned on