What were you trying to do that didn't work?

A customer found out that when accessing curl URL through httpd and mod_proxy, the first 2 responses were not containing Content-Type header.
The reason behind this seems to be a bug in curl when first initializing the connection.

Through stracing, we can see that the CGI handler initially sends Set-Cookie + Content-Type header.
Then subsequent requests made on the keepalived connection lead to CGI handler to only send Content-Type header.

Once the response sent by CGI handler is read by main curl daemon, the following code executes:

2367 /*
2368  * 'cupsdWriteClient()' - Write data to a client as needed.
2369  */
2370 
2371 void
2372 cupsdWriteClient(cupsd_client_t *con)   /* I - Client connection */
2373 {
 :
2490     if (con->pipe_pid && !con->got_fields)
2491     {
2492      /*
2493       * Inspect the data for Content-Type and other fields.
2494       */
2495 
2496       for (bufptr = con->header, bufend = con->header + con->header_used,
2497                field_col = 0;
2498            !con->got_fields && bufptr < bufend;
2499            bufptr ++)
2500       {
 :
2513           if (!con->sent_header)
2514           {
2515            /*
2516             * Handle redirection and CGI status codes...
2517             */
2518 
2519             http_field_t field;         /* HTTP field */
2520             char        *value = strchr(con->header, ':');
2521                                         /* Value of field */
2522 
2523             if (value)
2524             {
2525               *value++ = '\0';
2526               while (isspace(*value & 255))
2527                 value ++;
2528             }
2529 
2530             field = httpFieldValue(con->header);
2531 
2532             if (field != HTTP_FIELD_UNKNOWN && value)
2533             {
2534               httpSetField(con->http, field, value);
2535 
2536               if (field == HTTP_FIELD_LOCATION)
2537               {
2538                 con->pipe_status = HTTP_STATUS_SEE_OTHER;
2539                 con->sent_header = 2;
2540               }
2541               else
2542                 con->sent_header = 1;
2543             }
2544             else if (!_cups_strcasecmp(con->header, "Status") && value)
2545             {
2546               con->pipe_status = (http_status_t)atoi(value);
2547               con->sent_header = 2;
2548             }
2549             else if (!_cups_strcasecmp(con->header, "Set-Cookie") && value)
2550             {
2551               httpSetCookie(con->http, value);
2552               con->sent_header = 1;
2553             }
2554           }
 :

Initially a Set-Cookie: ...; Content-Type: ... is sent by CGI, this ends up extracting Set-Cookie and changing con->sent_header to 1 (lines 2551-2552).
Then the loop on headers (line 2496) for executes again and Content-Type is processed.
However, because con->sent_header on line 2513 is not 0 anymore, the header is discarded (line 2534 is not entered anymore).

Later, when more requests are made on the persistent connection, Content-Type is sent back, which causes the header to be processed correctly.

I don't understand the rationale behind this:

• The value con->sent_header can be set to 1 or 2, but none of the code seems to make a difference, the condition on the variable is always comparing to 0.
• Also, Set-Cookie seems to be considered as not a header, since httpSetCookie() is called instead of httpSetField(), so I would expect line 2552 to be removed.
• Finally setting this con->sent_header definitely makes all the other headers returned by CGI be ignored, I doubt this is expected.

This worked fine on RHEL7, the code was the same except there was no Set-Cookie handling, which must have made Content-Type header to be processed, whereas with RHEL8 and RHEL9 code, it's not the case.

Please provide the package NVR for which bug is seen:

cups-2.3.3op2-21.el9
cups-2.2.6-54.el8_9

How reproducible:

Always

Steps to reproduce

1. Install httpd, mod_ssl and curl

   # yum -y install httpd mod_ssl curl

1. Setup the proxy: /etc/httpd/conf.d/cups.conf

   DumpIOInput On
   DumpIOOutput Off
   LogLevel warn dumpio:trace7
   ProxyRequests Off
   SSLProxyEngine Off
   SSLProxyVerify None
   SSLProxyCheckPeerName Off
   SSLProxyCheckPeerExpire Off
   ProxyPass /example/cups/ http://localhost:631/ status=+I
   ProxyPassReverse /example/cups/ http://localhost:631/
   ProxyPassReverseCookiePath / /example/cups/

1. Start cups and restart httpd

   # systemctl restart cups httpd

1. Query cups through the proxy 3 times

   # for i in $(seq 1 3); do echo "Try #$i:"; curl -kv http://localhost/example/cups/admin/ 2>&1 | grep 'rel="stylesheet" href="'; sleep 1; done

1. Check httpd error log

   # grep "charset=utf-8" /etc/httpd/logs/error_log | grep mod_dumpio | grep -v DOCTYPE

Expected results

Seeing 3 times Content-Type in headers.

Actual results

Seeing only 1 time (the third request and onwards):

[Wed Mar 20 13:31:04.350017 2024] [dumpio:trace7] [pid 42406:tid 42564] mod_dumpio.c(100): [remote ::1:631] mod_dumpio:  dumpio_in (data-HEAP): ontent-Type: text/html;charset=utf-8\r\n