-
Story
-
Resolution: Unresolved
-
Undefined
-
None
-
None
-
None
-
False
-
-
False
-
-
Package 'hnswlib' does not build as-is via the AIPCC self-service pipeline and requires builder repository onboarding.
Build Failure Summary
Root Cause Analysis: `hnswlib` Build Failure
Summary
The build never reached compilation. It failed during the requirements parsing phase due to an invalid version specifier in the requirements file.
Root Cause
The file `/collection-repository/collections/torch-2.9.0/cpu-ubi9/requirements.txt` contains the entry:
hnswlib==any
The string `any` is not a valid PEP 440 version specifier. The Python `packaging` library correctly rejects it:
packaging._tokenizer.ParserSyntaxError: Expected semicolon (after name with no version specifier) or end
hnswlib==any
^
The `prepare-requirements` tool calls `Requirement("hnswlib==any")`, which raises an `InvalidRequirement` exception, and the entire bootstrap process aborts at line 0 of the requirements file.
Fix
Change the entry in `requirements.txt` from:
hnswlib==any
to simply:
hnswlib
An unconstrained package name (with no version specifier) already means "any version is acceptable," which is presumably the intended semantics. If a specific version is needed, use a valid PEP 440 version string (e.g., `hnswlib==0.8.0`).
Notes
- This is a requirements file syntax error, not a compilation or packaging issue with `hnswlib` itself.
- Because the failure occurs in `prepare-requirements` before `fromager` bootstrapping begins, no build artifacts (`build-sequence-summary.md`, logs, etc.) were produced — which matches the `WARNING: no matching files` messages in the artifact upload phase.
Packaging Analysis Summary
Here is the executive summary formatted as a JIRA comment in JIRA wiki markup:
Executive Summary: hnswlib Build Analysis
hnswlib (v0.8.0) is a C++ header-only approximate nearest neighbor search library with Python bindings via pybind11. It requires source compilation — no pre-built wheels exist on PyPI for any platform or version. The build itself is moderate complexity (4/10): a single C++ source file (
python_bindings/bindings.cpp
) compiled against pybind11 and numpy headers, with no external C/C++ library dependencies. The package is licensed under Apache-2.0, which is fully compatible with Red Hat redistribution requirements.
Three critical blockers must be resolved before building for enterprise distribution. First, the
PYBIND11_PLUGIN
macro used in
bindings.cpp:910
was removed in pybind11 3.x, and the pyproject.toml has no upper bound on pybind11 — the build will fail unless pybind11 is pinned to
<3.0
or the source is patched to use
PYBIND11_MODULE
. Second,
setup.py:57
references
setuptools.distutils.errors.CompileError
, an import path broken on Python 3.12+ — this must be patched. Third, the default
-march=native
compiler flag produces non-portable binaries that crash with "Illegal instruction" on CPUs different from the build host; the environment variable
HNSWLIB_NO_NATIVE=1
must be set for all distribution builds.
The recommended build approach is:
export HNSWLIB_NO_NATIVE=1 # Apply patches for pybind11 <3.0 pin and distutils import fix pip wheel . --no-build-isolation
For x86_64, the library provides runtime SIMD dispatch (SSE/AVX/AVX-512) via CPUID checks, so portable binaries still achieve optimal performance on capable hardware. For aarch64, no SIMD optimizations exist — the library falls back to scalar code with significantly lower performance. The upstream project is effectively unmaintained: 271 open issues, last release December 2023, and CI only tests up to Python 3.10.
Key stakeholder note: hnswlib is a CPU-only library with no CUDA or ROCm support. If the onboarding requirement (per AIPCC-10799) includes GPU-accelerated vector search indexes, hnswlib cannot satisfy that requirement. Alternatives such as faiss-gpu or NVIDIA cuVS/RAFT should be evaluated for GPU index targets. hnswlib can only serve the CPU index use case.
- is blocked by
-
AIPCC-10827 Add hnswlib into the RHAI pipeline onboarding collection
-
- In Progress
-
- mentioned on