Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-25409

Libserf >= 0.3.0 is broken, causing svn client to timeout

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • None
    • rhel-9.3.0
    • subversion
    • None
    • None
    • None
    • rhel-sst-cs-net-perf-services
    • ssg_core_services
    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • Red Hat Enterprise Linux
    • None
    • None
    • None
    • x86_64
    • None

       

      When trying to check out or update SVN repository, svn client occasionally crashes with error:

      [ondrejv@slsrvetxc-08 tmp]$ svn co https://svn.global.renesas.com/svntest/corp
      ...
      A    corp/silegoweb/system/libraries/Upload.php
      A    corp/silegoweb/system/libraries/Zip.php
      A    corp/silegoweb/system/libraries/cache/drivers/Cache_file.php
      A    corp/svn.ico
      svn: E175012: Connection timed out
      

      despite the fact there is no network timeout as packet trace shows all requests has been satisfied by the backend webserver. Also webserver and client are on the same network with no firewall or anything like that.

       

      The contributing factor for this to happen is if the svn server is behind a reverse proxy, for example in this setup:

       

      SSLProxyEngine On
      <VirtualHost slosvn001v:443>
          ServerName     slosvn001v.adwin.renesas.com
          SetEnv         proxy-nokeepalive 1    
          SSLCertificateKeyFile /etc/pki/tls/private/svn.global.renesas.com.key
          SSLCertificateFile /etc/pki/tls/certs/svn.global.renesas.com.crt
          SSLEngine on
          RewriteEngine   On
          RewriteMap      DS2Uid  "dbm:/svnroot/conf/ds2uidmap.db"
          RewriteMap      Uid2DS  "DBM:/svnroot/conf/uid2dsmap.db"       
      # This location rewrites Basic auth headers uid->sAMAccountName so it can be chewed by mod_auth_gssapi, it does not affect Negotiate
              <Location /svn>
      # Read Authorization header, grab the base64 authorization
                      RewriteCond %{HTTP:Authorization} ^Basic\ (.*)
                      RewriteRule .* - [E=X-B64:%1]
      # undecode header, map the user
                      RewriteCond expr "unbase64(%{ENV:X-B64}) =~ /(.*):(.*)/"
                      RewriteRule .* - [E=PROXY_USER:${Uid2DS:%1|%1},E=Password:%2]
      # encode header back into base64
                      RewriteCond expr "base64(%{ENV:PROXY_USER}.':'.%{ENV:Password}) =~ /(.*)/"
                      RewriteRule .* - [E=PROXY_HEADER:%1]
      # set back authorization header (if we have anything to set)
                      RequestHeader set Authorization "Basic %{PROXY_HEADER}e" env=X-B64
                      ProxyPass "http://localhost/svnauth"
                      ProxyPassReverse http://localhost/svnauth        
      </Location>
      
      </VirtualHost>
      <VirtualHost localhost:80>
              ServerName      slosvn001v.adwin.renesas.com
              ErrorLog        logs/svnerror.log
              TransferLog     logs/svnaccess.log
              LogFormat       "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b Auth: %{GSS_MECH}e"
      #        SetEnv          proxy-nokeepalive 1
      #  Fix Destination header for DAV_SVN so that "svn copy" does not crash with error 502
              RequestHeader edit Destination ^https http early
              RewriteEngine   On
              RewriteMap      DS2Uid  "dbm:/svnroot/conf/ds2uidmap.db"
              RemoteIPHeader  X-Forwarded-For
       
              <Directory /svnroot>
                      Options Indexes FollowSymLinks
                      AllowOverride None
                      Order allow,deny
                      Allow from all
              </Directory>
      # This location performs the actual authentication and also rewrites the auth headers back to UID so that SVN server receives UID and not SamAccountName (not much readable to human beings) 
              <Location /svnauth>
                      AuthType GSSAPI
                      AuthName "SVN Login"
                      GssapiCredStore         keytab:/etc/krb5.keytab
                      GssapiAllowedMech       krb5
                      GssapiBasicAuth         On
      # Not nice, but some  browsers behave strange when both Basic and Negotiate is offered
                      GssapiNegotiateOnce     On
      # Do not negotiate every request
                      GssapiConnectionBound   On
                      GssapiPublishMech       On
      # causes the authz file to be read from <repo path>/conf/authz    
                      require valid-user
      # set authorization header in form of base64(dstouidmap(<user>):\n). If map did not find the user, then unmapped user is used instead
                      RewriteCond expr "base64(%{REMOTE_USER}.':\n') =~ /(.*)/"
                      RewriteRule .* - [E=PROXY_USER:${DS2Uid:%{REMOTE_USER}|%1}]
                      RequestHeader set Authorization "Basic %{PROXY_USER}e"
                      ProxyPass "http://localhost/svn"
                      ProxyPassReverse http://localhost/svn        
              </Location>
      # This location is the actual SVN server - uses Anonymous provider as we do not really want to validate login here - we just want the %{REMOTE_USER} to be set
              <Location /svn>
                      AuthName "anonymous"
                      AuthType Basic
                      AuthBasicProvider anon
                      Anonymous "*"
                      require valid-user                
                      DAV svn
                      SVNParentPath /svnroot
                      SVNListParentPath On
                      SVNCacheFullTexts On
                      SVNCacheTextDeltas On
      #        SVNMasterURI https://svn.global.renesas.com/svn/
      # causes the authz file to be read from <repo path>/conf/authz
                      AuthzSVNReposRelativeAccessFile authz
              </Location>
      </VirtualHost>
       

      which is a setup for a Subversion server behind two reverse proxies - we are doing some authentication headers modification, but that's certainly not the reason for the timeout problem because:

       

      • packet trace does not show any problem (all request the svn client ever send, webserver responds)
      • no errors in the Apache logs to be found

      How reproducible:

      With 2 reverse proxies is the problem relatively easy to replicate - client just randomly times out. Sometimes also produces successful check out.

      If I use only one reverse proxy, the problem happens significantly less often, but happens.

      Steps to reproduce

      1. set up a webserver as described above (does not have to be exactly this way, important is the reverse proxy)
      2. issue 'svn co' or 'svn up'.
      3.  

      Expected results

      SVN client completes successfully

      Actual results

      SVN client crashes with a timeout "E175012: Connection timed out"

      Workaround

      Downgrade libserf to version 0.3.0 from RHEL-7.

      With libserf-0.3.0 the svn completes all tasks successfully, we can't replicate the problem

              tkorbar@redhat.com Tomáš Korbař
              ovalouse Ondrej Valousek (Inactive)
              Tomáš Korbař Tomáš Korbař
              rhel-cs-infra-services-qe rhel-cs-infra-services-qe rhel-cs-infra-services-qe rhel-cs-infra-services-qe
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: