Uploaded image for project: 'AMQ Clients'
  1. AMQ Clients
  2. ENTMQCL-1150

AMQP Python clients missing for Windows users or other users

    • Icon: Story Story
    • Resolution: Done
    • Icon: Major Major
    • 2.7.0.B1
    • 2.2.0.GA
    • amqp-python-client
    • None
    • Cabrillo

      In the download page there is not a amqp python client to download. [1]
      However, the way to download is via RHEL subscription management and RPM. [2]
      This is fine and dandy as long as the user is using RHEL.

      *Problems

      P1. However, most people will be developing using Windows or other OS. [3]
      P2. Why only python 2.7? what about Python3?
      P3. Ideally, python will prefer using pip or virtualenv to install packages. Is it possible to make install via pip? If so, can we have a document on how to do so?

      [1]
      https://access.redhat.com/jbossnetwork/restricted/listSoftware.html?product=jboss.amq.clients&downloadType=distributions&version=

      [2]
      https://access.redhat.com/documentation/en-us/red_hat_amq/7.2/html/using_the_amq_python_client/installation#installing_on_red_hat_enterprise_linux

      [3]
      https://access.redhat.com/documentation/en-us/red_hat_amq/7.2/html/using_the_amq_python_client/overview#supported_configurations
      AMQ Python supports the following OS and language versions:
      Red Hat Enterprise Linux 6 and 7 with Python 2.6 and 2.7
      Microsoft Windows Server 2012 R2 with Python 2.7

            [ENTMQCL-1150] AMQP Python clients missing for Windows users or other users

            Errata Tool added a comment -

            Since the problem described in this issue should be resolved in a recent advisory, it has been closed.

            For information on the advisory, and where to find the updated files, follow the link below.

            If the solution does not work for you, open a new bug report.
            https://access.redhat.com/errata/RHSA-2020:2605

            Errata Tool added a comment - Since the problem described in this issue should be resolved in a recent advisory, it has been closed. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:2605

            Jiri Daněk added a comment - - edited

            I had to install Python 3.6 (i did this in Python 3.6) and also Visual Studio. I used "developer command prompt" so that CMake finds msbuild [1]. The autodetected platform was 32 bit, so I had to give CMake correct platform using -G "Visual Studio 12 2013 Win64" (I think it actually was visual studio 2017, on my machine.) Other than that, I don't recall anything. What does it complain about, in your case? I can have a look tomorrow exactly what I did.

            Edit: because I did not change build type in CMake, it did Debug build, and at one point I had to reinstall my Python with debug libraries, so I rerun the installer, and in the "Change installation", I checked some debug versions of libraries, on the screen.

            [1] https://docs.microsoft.com/en-us/dotnet/framework/tools/developer-command-prompt-for-vs

            Jiri Daněk added a comment - - edited I had to install Python 3.6 (i did this in Python 3.6) and also Visual Studio. I used "developer command prompt" so that CMake finds msbuild [1] . The autodetected platform was 32 bit, so I had to give CMake correct platform using -G "Visual Studio 12 2013 Win64" (I think it actually was visual studio 2017, on my machine.) Other than that, I don't recall anything. What does it complain about, in your case? I can have a look tomorrow exactly what I did. Edit: because I did not change build type in CMake, it did Debug build, and at one point I had to reinstall my Python with debug libraries, so I rerun the installer, and in the "Change installation", I checked some debug versions of libraries, on the screen. [1] https://docs.microsoft.com/en-us/dotnet/framework/tools/developer-command-prompt-for-vs

            jdanek, did you need to install anything special to be able to build/install wheel on Windows?

            Irina Boverman (Inactive) added a comment - jdanek, did you need to install anything special to be able to build/install wheel on Windows?

            Jiri Daněk added a comment - - edited

            I had a look at pythonwheel.

            Currently, the CMake build of Qpid Proton puts together a build/python/dist directory, which contains python sources, swig .i file and C sources. The setup.py is then able to build a Linux pip package that either 1) relies on qpid-proton .so files in /usr/lib or builds its own .so file from the C sources which CMake copied there.

            Windows is not handled by setup.py, see https://issues.apache.org/jira/browse/PROTON-1467 ([python] setup.py script fails to build C sources on Windows) and https://issues.apache.org/jira/browse/PROTON-1468 ([python] attempting to pip install on windows should fail gracefully).

            What I tried is to put something into setup.py so that it will use the .so (actually DLL, I tried only Windows) produced by CMake and it will install them into lib/site-packages in python virtualenv.

            Index: python/setup.py.in
            ===================================================================
            --- python/setup.py.in	(revision 6397116291fd569ce03025c1b3a135172de3f2a6)
            +++ python/setup.py.in	(date 1547492834727)
            @@ -50,7 +50,7 @@
             import distutils.spawn as ds_spawn
             import distutils.sysconfig as ds_sys
             from distutils.ccompiler import new_compiler, get_default_compiler
            -from distutils.core import setup, Extension
            +from setuptools import setup, Extension
             from distutils.command.build import build
             from distutils.command.build_ext import build_ext
             from distutils.command.sdist import sdist
            @@ -262,14 +262,38 @@
                     ldirs = misc.pkg_config_get_var('libqpid-proton', 'libdir')
                     _cproton.library_dirs.extend(ldirs.split())
             
            +    def use_compiled_proton(self):
            +        """The Proton development headers and library are installed, update the
            +        _cproton extension to tell it where to find the library and headers.
            +        """
            +        # update the Extension instance passed to setup() to use the installed
            +        # headers and link library
            +        _cproton = self.distribution.ext_modules[-1]
            +        # incs = misc.pkg_config_get_var('libqpid-proton', 'includedir')
            +        incs = r"C:\Users\Vitorio\CLionProjects\qpid-proton\build\c\include"
            +        for i in incs.split():
            +            _cproton.swig_opts.append('-I%s' % i)
            +            _cproton.include_dirs.append(i)
            +        # ldirs = misc.pkg_config_get_var('libqpid-proton', 'libdir')
            +        ldirs = r"C:\Users\Vitorio\CLionProjects\qpid-proton\build\c\Debug"
            +        _cproton.library_dirs.extend(ldirs.split())
            +
            +        # https://docs.python.org/2/distutils/setupscript.html#installing-additional-files
            +        self.distribution.data_files = [('lib/site-packages', [
            +            r'C:\Users\Vitorio\CLionProjects\qpid-proton\build\c\Debug\qpid-proton-cored.dll',
            +            r'C:\Users\Vitorio\CLionProjects\qpid-proton\build\c\Debug\qpid-proton-proactord.dll',
            +            r'C:\Users\Vitorio\CLionProjects\qpid-proton\build\c\Debug\qpid-protond.dll',
            +        ])]
            +
                 def run(self):
                     # check if the Proton library and headers are installed and are
                     # compatible with this version of the binding.
            -        if self.libqpid_proton_installed(_PROTON_VERSION_STR):
            -            self.use_installed_proton()
            -        else:
            -            # Proton not installed or compatible, use bundled proton-c sources
            -            self.use_bundled_proton()
            +        # if self.libqpid_proton_installed(_PROTON_VERSION_STR):
            +        #     self.use_installed_proton()
            +        # else:
            +        #    # Proton not installed or compatible, use bundled proton-c sources
            +        #    self.use_bundled_proton()
            +        self.use_compiled_proton()
                     self.prepare_swig_wrap()
             
             
            @@ -330,4 +354,4 @@
                                          sources=['cproton.i', 'cproton_wrap.c'],
                                          swig_opts=['-threads'],
                                          extra_compile_args=['-pthread'],
            -                             libraries=['qpid-proton'])])
            +                             libraries=['qpid-protond'])])
            

            python setup.py bdist_wheel

            pip install python_qpid_proton-0.27.0-cp37-cp37m-win_amd64.whl

            There are some hardcoded paths in that, it assumes a Debug CMake build, and I did not try to figure out the logic when to 1) compile from source 2) compile against system 3) use dlls build by CMake.

            There would need to be a .wheel file for every combination of python version (2.7, 3.5, ...) and architecture (32, 64bit).

            Jiri Daněk added a comment - - edited I had a look at pythonwheel. Currently, the CMake build of Qpid Proton puts together a build/python/dist directory, which contains python sources, swig .i file and C sources. The setup.py is then able to build a Linux pip package that either 1) relies on qpid-proton .so files in /usr/lib or builds its own .so file from the C sources which CMake copied there. Windows is not handled by setup.py, see https://issues.apache.org/jira/browse/PROTON-1467 ( [python] setup.py script fails to build C sources on Windows) and https://issues.apache.org/jira/browse/PROTON-1468 ( [python] attempting to pip install on windows should fail gracefully). What I tried is to put something into setup.py so that it will use the .so (actually DLL, I tried only Windows) produced by CMake and it will install them into lib/site-packages in python virtualenv. Index: python/setup.py.in =================================================================== --- python/setup.py.in (revision 6397116291fd569ce03025c1b3a135172de3f2a6) +++ python/setup.py.in (date 1547492834727) @@ -50,7 +50,7 @@ import distutils.spawn as ds_spawn import distutils.sysconfig as ds_sys from distutils.ccompiler import new_compiler, get_default_compiler -from distutils.core import setup, Extension +from setuptools import setup, Extension from distutils.command.build import build from distutils.command.build_ext import build_ext from distutils.command.sdist import sdist @@ -262,14 +262,38 @@ ldirs = misc.pkg_config_get_var( 'libqpid-proton' , 'libdir' ) _cproton.library_dirs.extend(ldirs.split()) + def use_compiled_proton(self): + """The Proton development headers and library are installed, update the + _cproton extension to tell it where to find the library and headers. + """ + # update the Extension instance passed to setup() to use the installed + # headers and link library + _cproton = self.distribution.ext_modules[-1] + # incs = misc.pkg_config_get_var( 'libqpid-proton' , 'includedir' ) + incs = r "C:\Users\Vitorio\CLionProjects\qpid-proton\build\c\include" + for i in incs.split(): + _cproton.swig_opts.append( '-I%s' % i) + _cproton.include_dirs.append(i) + # ldirs = misc.pkg_config_get_var( 'libqpid-proton' , 'libdir' ) + ldirs = r "C:\Users\Vitorio\CLionProjects\qpid-proton\build\c\Debug" + _cproton.library_dirs.extend(ldirs.split()) + + # https: //docs.python.org/2/distutils/setupscript.html#installing-additional-files + self.distribution.data_files = [( 'lib/site-packages' , [ + r 'C:\Users\Vitorio\CLionProjects\qpid-proton\build\c\Debug\qpid-proton-cored.dll' , + r 'C:\Users\Vitorio\CLionProjects\qpid-proton\build\c\Debug\qpid-proton-proactord.dll' , + r 'C:\Users\Vitorio\CLionProjects\qpid-proton\build\c\Debug\qpid-protond.dll' , + ])] + def run(self): # check if the Proton library and headers are installed and are # compatible with this version of the binding. - if self.libqpid_proton_installed(_PROTON_VERSION_STR): - self.use_installed_proton() - else : - # Proton not installed or compatible, use bundled proton-c sources - self.use_bundled_proton() + # if self.libqpid_proton_installed(_PROTON_VERSION_STR): + # self.use_installed_proton() + # else : + # # Proton not installed or compatible, use bundled proton-c sources + # self.use_bundled_proton() + self.use_compiled_proton() self.prepare_swig_wrap() @@ -330,4 +354,4 @@ sources=[ 'cproton.i' , 'cproton_wrap.c' ], swig_opts=[ '-threads' ], extra_compile_args=[ '-pthread' ], - libraries=[ 'qpid-proton' ])]) + libraries=[ 'qpid-protond' ])]) python setup.py bdist_wheel pip install python_qpid_proton-0.27.0-cp37-cp37m-win_amd64.whl There are some hardcoded paths in that, it assumes a Debug CMake build, and I did not try to figure out the logic when to 1) compile from source 2) compile against system 3) use dlls build by CMake. There would need to be a .wheel file for every combination of python version (2.7, 3.5, ...) and architecture (32, 64bit).

            Jiri Daněk added a comment -

            The python client is in "Red Hat AMQ C++ Client 2.2.0 Windows SDK" portal download, under lib/proton/bindings/python. I did not found any documentation for this myself, now. rkubis_kafka_devexp, do you know of any docs?

            There is ticket for pip support already, ENTMQCL-832. rhn-support-whui Are you aware of some other Red Hat product which ships for pip?

            There is also ticket for Python 3 on Windows, ENTMQCL-797, there was issue reported, which needed to be fixed ENTMQCL-761.

            Jiri Daněk added a comment - The python client is in "Red Hat AMQ C++ Client 2.2.0 Windows SDK" portal download, under lib/proton/bindings/python . I did not found any documentation for this myself, now. rkubis_kafka_devexp , do you know of any docs? There is ticket for pip support already, ENTMQCL-832. rhn-support-whui Are you aware of some other Red Hat product which ships for pip? There is also ticket for Python 3 on Windows, ENTMQCL-797 , there was issue reported, which needed to be fixed ENTMQCL-761 .

              jross@redhat.com Justin Ross
              rhn-support-whui Roger Hui
              Radim Kubis Radim Kubis
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: