Uploaded image for project: 'AI Platform Core Components'
  1. AI Platform Core Components
  2. AIPCC-838

builder cannot handle neural-compressor-pt

    • Icon: Epic Epic
    • Resolution: Done
    • Icon: Undefined Undefined
    • None
    • None
    • Development Platform
    • None
    • builder fix neural-compressor-pt
    • False
    • Hide

      None

      Show
      None
    • False
    • To Do
    • AIPCC-680 - Build vllm With Support for Audio and Video Models
    • AIPCC-680Build vllm With Support for Audio and Video Models

      Trying to test neural-compressor-pt as a pre-built wheel causes an error because it wants an old numpy and setuptools.

      DEBUG:fromager.external_commands:73: running: _PYPROJECT_HOOKS_BUILD_BACKEND=setuptools.build_meta _PYPROJECT_HOOKS_BACKEND_PATH=/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0 MAKEFLAGS=-j192 CMAKE_BUILD_PARALLEL_LEVEL=192 MAX_JOBS=192 /opt/app-root/bin/python3.11 /opt/app-root/lib64/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp5_ua33uo in /work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0
      ERROR:fromager.external_commands:102: ['/opt/app-root/bin/python3.11', '/opt/app-root/lib64/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py', 'get_requires_for_build_wheel', '/tmp/tmp5_ua33uo'] failed with running egg_info
      writing setuptools.egg-info/PKG-INFO
      writing dependency_links to setuptools.egg-info/dependency_links.txt
      writing entry points to setuptools.egg-info/entry_points.txt
      writing requirements to setuptools.egg-info/requires.txt
      writing top-level names to setuptools.egg-info/top_level.txt
      Traceback (most recent call last):
        File "/opt/app-root/lib64/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
          main()
        File "/opt/app-root/lib64/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in main
          json_out["return_val"] = hook(**hook_input["kwargs"])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 143, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/build_meta.py", line 162, in get_requires_for_build_wheel
          return self._get_build_requires(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/build_meta.py", line 143, in _get_build_requires
          self.run_setup()
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/build_meta.py", line 158, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 87, in <module>
          dist = setuptools.setup(**setup_params)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/_distutils/core.py", line 148, in setup
          dist.run_commands()
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/_distutils/dist.py", line 967, in run_commands
          self.run_command(cmd)
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/command/egg_info.py", line 299, in run
          self.find_sources()
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/command/egg_info.py", line 306, in find_sources
          mm.run()
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/command/egg_info.py", line 541, in run
          self.add_defaults()
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/command/egg_info.py", line 578, in add_defaults
          sdist.add_defaults(self)
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/_distutils/command/sdist.py", line 226, in add_defaults
          self._add_defaults_python()
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/command/sdist.py", line 111, in _add_defaults_python
          build_py = self.get_finalized_command('build_py')
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/_distutils/cmd.py", line 299, in get_finalized_command
          cmd_obj.ensure_finalized()
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/_distutils/cmd.py", line 107, in ensure_finalized
          self.finalize_options()
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/command/build_py.py", line 29, in finalize_options
          orig.build_py.finalize_options(self)
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/_distutils/command/build_py.py", line 43, in finalize_options
          self.set_undefined_options('build',
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/_distutils/cmd.py", line 286, in set_undefined_options
          src_cmd_obj = self.distribution.get_command_obj(src_cmd)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/_distutils/dist.py", line 858, in get_command_obj
          klass = self.get_command_class(command)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/setuptools/dist.py", line 887, in get_command_class
          self.cmdclass[command] = cmdclass = ep.load()
                                              ^^^^^^^^^
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/pkg_resources/__init__.py", line 2464, in load
          return self.resolve()
                 ^^^^^^^^^^^^^^
        File "/work/bootstrap-output.gaudi-ubi9/work-dir/setuptools-59.2.0/setuptools-59.2.0/pkg_resources/__init__.py", line 2470, in resolve
          module = __import__(self.module_name, fromlist=['__name__'], level=0)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      ModuleNotFoundError: No module named 'setuptools.command.build' 84%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏                          | 86/102 [00:46<00:08,  1.84pkg/s]
      DEBUG:fromager.__main__:245: could not handle build-system dependency setuptools==59.2.0 (59.2.0) for install dependency numpy==1.23.5; python_version < "3.12" (1.23.5) for toplevel dependency neural-compressor-pt (3.2)
      Traceback (most recent call last):
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/bootstrapper.py", line 348, in _handle_build_requirements
          resolved = self.bootstrap(req=dep, req_type=build_type)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/bootstrapper.py", line 157, in bootstrap
          build_dependencies = self._prepare_build_dependencies(req, sdist_root_dir)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/bootstrapper.py", line 319, in _prepare_build_dependencies
          build_backend_dependencies = dependencies.get_build_backend_dependencies(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/dependencies.py", line 112, in get_build_backend_dependencies
          orig_deps = overrides.find_and_invoke(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/overrides.py", line 52, in find_and_invoke
          result = invoke(fn, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/overrides.py", line 71, in invoke
          return fn(**kwargs)
                 ^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/dependencies.py", line 147, in default_get_build_backend_dependencies
          return hook_caller.get_requires_for_build_wheel()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/pyproject_hooks/_impl.py", line 196, in get_requires_for_build_wheel
          return self._call_hook(
                 ^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/pyproject_hooks/_impl.py", line 392, in _call_hook
          self._subprocess_runner(
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/dependencies.py", line 321, in _run_hook_with_extra_environ
          external_commands.run(
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/external_commands.py", line 115, in run
          raise err_type(completed.returncode, cmd, output)
      subprocess.CalledProcessError: Command '['/opt/app-root/bin/python3.11', '/opt/app-root/lib64/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py', 'get_requires_for_build_wheel', '/tmp/tmp5_ua33uo']' returned non-zero exit status 1.The above exception was the direct cause of the following exception:Traceback (most recent call last):
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/bootstrapper.py", line 230, in bootstrap
          self.bootstrap(dep, child_req_type)
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/bootstrapper.py", line 157, in bootstrap
          build_dependencies = self._prepare_build_dependencies(req, sdist_root_dir)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/bootstrapper.py", line 314, in _prepare_build_dependencies
          self._handle_build_requirements(
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/bootstrapper.py", line 350, in _handle_build_requirements
          raise ValueError(f"could not handle {self._explain}") from err
      ValueError: could not handle build-system dependency setuptools==59.2.0 (59.2.0) for install dependency numpy==1.23.5; python_version < "3.12" (1.23.5) for toplevel dependency neural-compressor-pt (3.2)The above exception was the direct cause of the following exception:Traceback (most recent call last):
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/__main__.py", line 243, in invoke_main
          main(auto_envvar_prefix="FROMAGER")
        File "/opt/app-root/lib64/python3.11/site-packages/click/core.py", line 1161, in __call__
          return self.main(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/click/core.py", line 1082, in main
          rv = self.invoke(ctx)
               ^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/click/core.py", line 1697, in invoke
          return _process_result(sub_ctx.command.invoke(sub_ctx))
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/click/core.py", line 1443, in invoke
          return ctx.invoke(self.callback, **ctx.params)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/click/core.py", line 788, in invoke
          return __callback(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/click/decorators.py", line 45, in new_func
          return f(get_current_context().obj, *args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/commands/bootstrap.py", line 172, in bootstrap
          bt.bootstrap(req, requirements_file.RequirementType.TOP_LEVEL)
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/bootstrapper.py", line 232, in bootstrap
          raise ValueError(f"could not handle {self._explain}") from err
      ValueError: could not handle build-system dependency setuptools==59.2.0 (59.2.0) for install dependency numpy==1.23.5; python_version < "3.12" (1.23.5) for toplevel dependency neural-compressor-pt (3.2)
      ERROR:fromager.__main__:249: ERROR: could not handle build-system dependency setuptools==59.2.0 (59.2.0) for install dependency numpy==1.23.5; python_version < "3.12" (1.23.5) for toplevel dependency neural-compressor-pt (3.2) because could not handle build-system dependency setuptools==59.2.0 (59.2.0) for install dependency numpy==1.23.5; python_version < "3.12" (1.23.5) for toplevel dependency neural-compressor-pt (3.2) because Command '['/opt/app-root/bin/python3.11', '/opt/app-root/lib64/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py', 'get_requires_for_build_wheel', '/tmp/tmp5_ua33uo']' returned non-zero exit status 1.
      Traceback (most recent call last):
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/bootstrapper.py", line 348, in _handle_build_requirements
          resolved = self.bootstrap(req=dep, req_type=build_type)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/bootstrapper.py", line 157, in bootstrap
          build_dependencies = self._prepare_build_dependencies(req, sdist_root_dir)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/bootstrapper.py", line 319, in _prepare_build_dependencies
          build_backend_dependencies = dependencies.get_build_backend_dependencies(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/dependencies.py", line 112, in get_build_backend_dependencies
          orig_deps = overrides.find_and_invoke(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/overrides.py", line 52, in find_and_invoke
          result = invoke(fn, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/overrides.py", line 71, in invoke
          return fn(**kwargs)
                 ^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/dependencies.py", line 147, in default_get_build_backend_dependencies
          return hook_caller.get_requires_for_build_wheel()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/pyproject_hooks/_impl.py", line 196, in get_requires_for_build_wheel
          return self._call_hook(
                 ^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/pyproject_hooks/_impl.py", line 392, in _call_hook
          self._subprocess_runner(
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/dependencies.py", line 321, in _run_hook_with_extra_environ
          external_commands.run(
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/external_commands.py", line 115, in run
          raise err_type(completed.returncode, cmd, output)
      subprocess.CalledProcessError: Command '['/opt/app-root/bin/python3.11', '/opt/app-root/lib64/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py', 'get_requires_for_build_wheel', '/tmp/tmp5_ua33uo']' returned non-zero exit status 1.The above exception was the direct cause of the following exception:Traceback (most recent call last):
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/bootstrapper.py", line 230, in bootstrap
          self.bootstrap(dep, child_req_type)
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/bootstrapper.py", line 157, in bootstrap
          build_dependencies = self._prepare_build_dependencies(req, sdist_root_dir)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/bootstrapper.py", line 314, in _prepare_build_dependencies
          self._handle_build_requirements(
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/bootstrapper.py", line 350, in _handle_build_requirements
          raise ValueError(f"could not handle {self._explain}") from err
      ValueError: could not handle build-system dependency setuptools==59.2.0 (59.2.0) for install dependency numpy==1.23.5; python_version < "3.12" (1.23.5) for toplevel dependency neural-compressor-pt (3.2)The above exception was the direct cause of the following exception:Traceback (most recent call last):
        File "/opt/app-root/bin/fromager", line 8, in <module>
          sys.exit(invoke_main())
                   ^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/__main__.py", line 243, in invoke_main
          main(auto_envvar_prefix="FROMAGER")
        File "/opt/app-root/lib64/python3.11/site-packages/click/core.py", line 1161, in __call__
          return self.main(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/click/core.py", line 1082, in main
          rv = self.invoke(ctx)
               ^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/click/core.py", line 1697, in invoke
          return _process_result(sub_ctx.command.invoke(sub_ctx))
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/click/core.py", line 1443, in invoke
          return ctx.invoke(self.callback, **ctx.params)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/click/core.py", line 788, in invoke
          return __callback(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/click/decorators.py", line 45, in new_func
          return f(get_current_context().obj, *args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/commands/bootstrap.py", line 172, in bootstrap
          bt.bootstrap(req, requirements_file.RequirementType.TOP_LEVEL)
        File "/opt/app-root/lib64/python3.11/site-packages/fromager/bootstrapper.py", line 232, in bootstrap
          raise ValueError(f"could not handle {self._explain}") from err
      ValueError: could not handle build-system dependency setuptools==59.2.0 (59.2.0) for install dependency numpy==1.23.5; python_version < "3.12" (1.23.5) for toplevel dependency neural-compressor-pt (3.2)

              Unassigned Unassigned
              dhellman@redhat.com Doug Hellmann
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: