Uploaded image for project: 'Insights Experiences'
  1. Insights Experiences
  2. HMS-3049

[QE] HMS integration pipeline failing - ssh to ec2 instances

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Critical Critical
    • hms-1.4
    • None
    • Conversions, QE
    • None

      ---------------------------- live log sessionfinish ----------------------------
      2023-11-15 12:05:34 INFO Available auth types: ['basic', 'cert', 'identity', 'jwt', 'uht']
      2023-11-15 12:05:34 INFO Setting auth_type to jwt
      2023-11-15 12:05:34 INFO http_client: Setting maximum number of retries of each connection to 3
      Saved results archive to e09a1400-10f7-402d-917c-8d94c25c91e3.tar.gz
      Ibutsu server: https://ibutsu-api.apps.ocp4.prod.psi.redhat.com/
      
      
      =================================== FAILURES ===================================
      _ test_admin_aws_azure_gcp_e2e[False-us-central1-a-eastus-us-east-1-bulk_create_stacks0-rhel-9-2048] _
      
      self = ProvisionedHost(user='ec2-user', hostname='10.31.62.214', private_key_file='/tmp/iqe-4sbl_0qd/9GQvmdVwRz_jJIqaF', timeout=360, client=<paramiko.client.SSHClient object at 0x7f868f92fbb0>, auto_registration=False)
      max_timeout = 360, _cur_timeout = 1
      
          def establish_connection(self, max_timeout, _cur_timeout=1):
              # From https://github.com/SatelliteQE/broker/blob/master/broker/helpers.py#L385
              try:
      >           self.client.connect(
                      hostname=self.hostname,
                      username=self.user,
                      key_filename=self.private_key_file,
                  )
      
      /iqe_venv/lib/python3.9/site-packages/iqe_provisioning/fixtures/helpers.py:95: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      self = <paramiko.client.SSHClient object at 0x7f868f92fbb0>
      hostname = '10.31.62.214', port = 22, username = 'ec2-user', password = None
      pkey = None, key_filename = '/tmp/iqe-4sbl_0qd/9GQvmdVwRz_jJIqaF'
      timeout = None, allow_agent = True, look_for_keys = True, compress = False
      sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0>
      gss_auth = False, gss_kex = False, gss_deleg_creds = True, gss_host = None
      banner_timeout = None, auth_timeout = None, channel_timeout = None
      gss_trust_dns = True, passphrase = None, disabled_algorithms = None
      transport_factory = None, auth_strategy = None
      
          def connect(
              self,
              hostname,
              port=SSH_PORT,
              username=None,
              password=None,
              pkey=None,
              key_filename=None,
              timeout=None,
              allow_agent=True,
              look_for_keys=True,
              compress=False,
              sock=None,
              gss_auth=False,
              gss_kex=False,
              gss_deleg_creds=True,
              gss_host=None,
              banner_timeout=None,
              auth_timeout=None,
              channel_timeout=None,
              gss_trust_dns=True,
              passphrase=None,
              disabled_algorithms=None,
              transport_factory=None,
              auth_strategy=None,
          ):
              """
              Connect to an SSH server and authenticate to it.  The server's host key
              is checked against the system host keys (see `load_system_host_keys`)
              and any local host keys (`load_host_keys`).  If the server's hostname
              is not found in either set of host keys, the missing host key policy
              is used (see `set_missing_host_key_policy`).  The default policy is
              to reject the key and raise an `.SSHException`.
          
              Authentication is attempted in the following order of priority:
          
                  - The ``pkey`` or ``key_filename`` passed in (if any)
          
                    - ``key_filename`` may contain OpenSSH public certificate paths
                      as well as regular private-key paths; when files ending in
                      ``-cert.pub`` are found, they are assumed to match a private
                      key, and both components will be loaded. (The private key
                      itself does *not* need to be listed in ``key_filename`` for
                      this to occur - *just* the certificate.)
          
                  - Any key we can find through an SSH agent
                  - Any "id_rsa", "id_dsa" or "id_ecdsa" key discoverable in
                    ``~/.ssh/``
          
                    - When OpenSSH-style public certificates exist that match an
                      existing such private key (so e.g. one has ``id_rsa`` and
                      ``id_rsa-cert.pub``) the certificate will be loaded alongside
                      the private key and used for authentication.
          
                  - Plain username/password auth, if a password was given
          
              If a private key requires a password to unlock it, and a password is
              passed in, that password will be used to attempt to unlock the key.
          
              :param str hostname: the server to connect to
              :param int port: the server port to connect to
              :param str username:
                  the username to authenticate as (defaults to the current local
                  username)
              :param str password:
                  Used for password authentication; is also used for private key
                  decryption if ``passphrase`` is not given.
              :param str passphrase:
                  Used for decrypting private keys.
              :param .PKey pkey: an optional private key to use for authentication
              :param str key_filename:
                  the filename, or list of filenames, of optional private key(s)
                  and/or certs to try for authentication
              :param float timeout:
                  an optional timeout (in seconds) for the TCP connect
              :param bool allow_agent:
                  set to False to disable connecting to the SSH agent
              :param bool look_for_keys:
                  set to False to disable searching for discoverable private key
                  files in ``~/.ssh/``
              :param bool compress: set to True to turn on compression
              :param socket sock:
                  an open socket or socket-like object (such as a `.Channel`) to use
                  for communication to the target host
              :param bool gss_auth:
                  ``True`` if you want to use GSS-API authentication
              :param bool gss_kex:
                  Perform GSS-API Key Exchange and user authentication
              :param bool gss_deleg_creds: Delegate GSS-API client credentials or not
              :param str gss_host:
                  The targets name in the kerberos database. default: hostname
              :param bool gss_trust_dns:
                  Indicates whether or not the DNS is trusted to securely
                  canonicalize the name of the host being connected to (default
                  ``True``).
              :param float banner_timeout: an optional timeout (in seconds) to wait
                  for the SSH banner to be presented.
              :param float auth_timeout: an optional timeout (in seconds) to wait for
                  an authentication response.
              :param float channel_timeout: an optional timeout (in seconds) to wait
                   for a channel open response.
              :param dict disabled_algorithms:
                  an optional dict passed directly to `.Transport` and its keyword
                  argument of the same name.
              :param transport_factory:
                  an optional callable which is handed a subset of the constructor
                  arguments (primarily those related to the socket, GSS
                  functionality, and algorithm selection) and generates a
                  `.Transport` instance to be used by this client. Defaults to
                  `.Transport.__init__`.
              :param auth_strategy:
                  an optional instance of `.AuthStrategy`, triggering use of this
                  newer authentication mechanism instead of SSHClient's legacy auth
                  method.
          
                  .. warning::
                      This parameter is **incompatible** with all other
                      authentication-related parameters (such as, but not limited to,
                      ``password``, ``key_filename`` and ``allow_agent``) and will
                      trigger an exception if given alongside them.
          
              :returns:
                  `.AuthResult` if ``auth_strategy`` is non-``None``; otherwise,
                  returns ``None``.
          
              :raises BadHostKeyException:
                  if the server's host key could not be verified.
              :raises AuthenticationException:
                  if authentication failed.
              :raises UnableToAuthenticate:
                  if authentication failed (when ``auth_strategy`` is non-``None``;
                  and note that this is a subclass of ``AuthenticationException``).
              :raises socket.error:
                  if a socket error (other than connection-refused or
                  host-unreachable) occurred while connecting.
              :raises NoValidConnectionsError:
                  if all valid connection targets for the requested hostname (eg IPv4
                  and IPv6) yielded connection-refused or host-unreachable socket
                  errors.
              :raises SSHException:
                  if there was any other error connecting or establishing an SSH
                  session.
          
              .. versionchanged:: 1.15
                  Added the ``banner_timeout``, ``gss_auth``, ``gss_kex``,
                  ``gss_deleg_creds`` and ``gss_host`` arguments.
              .. versionchanged:: 2.3
                  Added the ``gss_trust_dns`` argument.
              .. versionchanged:: 2.4
                  Added the ``passphrase`` argument.
              .. versionchanged:: 2.6
                  Added the ``disabled_algorithms`` argument.
              .. versionchanged:: 2.12
                  Added the ``transport_factory`` argument.
              .. versionchanged:: 3.2
                  Added the ``auth_strategy`` argument.
              """
              if not sock:
                  errors = {}
                  # Try multiple possible address families (e.g. IPv4 vs IPv6)
                  to_try = list(self._families_and_addresses(hostname, port))
                  for af, addr in to_try:
                      try:
                          sock = socket.socket(af, socket.SOCK_STREAM)
                          if timeout is not None:
                              try:
                                  sock.settimeout(timeout)
                              except:
                                  pass
      >                   sock.connect(addr)
      E                   TimeoutError: [Errno 110] Connection timed out
      

              pakotvan@redhat.com Pavol Kotvan
              pakotvan@redhat.com Pavol Kotvan
              None
              None
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: