Uploaded image for project: 'Red Hat 3scale API Management'
  1. Red Hat 3scale API Management
  2. THREESCALE-7514

Nginx filter policy is not working as expected when using content-caching

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Won't Do
    • Icon: Major Major
    • None
    • None
    • Gateway
    • False
    • False
    • Not Started
    • Not Started
    • Not Started
    • Not Started
    • Not Started
    • Not Started
    • +

      When content-caching is defined, the If-match header is always used, so the
      test[0] without it, it's working correctly, but when content-caching is enabled,
      the nginx-filter is not working as expected:

      A sample scenario is like this:

      master_process off;
      worker_processes  1;
      daemon off;
      error_log /dev/stdout debug;
      
      events {
        accept_mutex off;
        worker_connections 5000;
      }
      
      
      
      http {
          proxy_cache_path /tmp/cache levels=1:2 keys_zone=apicast_cache:10m;
      
          upstream backend {
              server 127.0.0.1:8081;
              balancer_by_lua_block {
                ngx.log(ngx.ERR, "Setting header on balancer")
                ngx.req.set_header("if-match", "anything")
              }
              keepalive 10;  # connection pool
          }
      
          server {
              access_log  off;
              listen 8081;
              resolver 8.8.8.8;
      
              location / {
                rewrite_by_lua_block {
                  local h = ngx.req.get_headers()
                  ngx.req.clear_header("if-match")
                  ngx.say(require("cjson").encode(h))
                }
              }
          }
      
          server {
      
            access_log  off;
            listen 8080;
            resolver 8.8.8.8;
      
            location / {
      
                rewrite_by_lua_block {
                  ngx.req.clear_header("if-match")
                }
      
                content_by_lua_block {
                   ngx.exec("@upstream")
                }
            }
      
            location @upstream {
      
              set $cache_request 'false';
              set $cache_zone 'off';
      
              proxy_cache apicast_cache;
              proxy_cache_key $scheme$request_method$proxy_host$request_uri;
              proxy_no_cache "false";
              proxy_cache_valid 200 302 1m;
      
              add_header X-Cache-Status $upstream_cache_status;
      
              proxy_pass http://backend;
              proxy_http_version 1.1;
            }
          }
      }
      
      
      

      So, when doing the send header, it'll use the
      ngx_http_not_modified_header_filter, and things will mess up when
      content-caching is enabled.

      The backtrace looks like this when is set, and the main issue is the
      `ngx_http_send_header` function, that applies the filter that are defined on the
      `ngx_http_not_modified_filter_module.c`:

      (gdb) bt
      #0  ngx_http_not_modified_header_filter (r=0x55c21900e990) at src/http/modules/ngx_http_not_modified_filter_module.c:71
      #1  ngx_http_not_modified_header_filter (r=0x55c21900e990) at src/http/modules/ngx_http_not_modified_filter_module.c:55
      #2  0x000055c216ff8069 in ngx_http_lua_send_header_if_needed (ctx=0x55c21900f780, r=<optimized out>) at ../ngx_lua-0.10.19/src/ngx_http_lua_util.c:549
      #3  ngx_http_lua_send_header_if_needed (r=<optimized out>, ctx=<optimized out>) at ../ngx_lua-0.10.19/src/ngx_http_lua_util.c:524
      #4  0x000055c216ff80e8 in ngx_http_lua_send_chain_link (r=r@entry=0x55c21900e990, ctx=ctx@entry=0x55c21900f780, in=in@entry=0x55c21900fd40)
          at ../ngx_lua-0.10.19/src/ngx_http_lua_util.c:591
      #5  0x000055c216ff8327 in ngx_http_lua_send_chain_link (r=r@entry=0x55c21900e990, ctx=ctx@entry=0x55c21900f780, in=in@entry=0x55c21900fd40)
          at ../ngx_lua-0.10.19/src/ngx_http_lua_util.c:569
      #6  0x000055c216fec479 in ngx_http_lua_ngx_echo (L=0x7f4f3d2a7ab0, newline=<optimized out>) at ../ngx_lua-0.10.19/src/ngx_http_lua_output.c:239
      #7  0x00007f4f3d726f75 in lj_BC_FUNCC () from /usr/local/openresty/luajit/lib/libluajit-5.1.so.2
      #8  0x000055c216ff9a52 in ngx_http_lua_run_thread (L=L@entry=0x7f4f3d2d0380, r=r@entry=0x55c21900e990, ctx=ctx@entry=0x55c21900f780, nrets=<optimized out>,
          nrets@entry=0) at ../ngx_lua-0.10.19/src/ngx_http_lua_util.c:1167
      #9  0x000055c216ffd7a8 in ngx_http_lua_rewrite_by_chunk (L=0x7f4f3d2d0380, r=0x55c21900e990) at ../ngx_lua-0.10.19/src/ngx_http_lua_rewriteby.c:337
      #10 0x000055c216ffd9ee in ngx_http_lua_rewrite_handler (r=0x55c21900e990) at ../ngx_lua-0.10.19/src/ngx_http_lua_rewriteby.c:162
      #11 0x000055c216f48b9d in ngx_http_core_rewrite_phase (r=0x55c21900e990, ph=<optimized out>) at src/http/ngx_http_core_module.c:932
      #12 0x000055c216f443e5 in ngx_http_core_run_phases (r=r@entry=0x55c21900e990) at src/http/ngx_http_core_module.c:878
      #13 0x000055c216f444a4 in ngx_http_handler (r=r@entry=0x55c21900e990) at src/http/ngx_http_core_module.c:861
      #14 0x000055c216f4f1c5 in ngx_http_process_request (r=r@entry=0x55c21900e990) at src/http/ngx_http_request.c:2081
      #15 0x000055c216f4f747 in ngx_http_process_request_headers (rev=<optimized out>) at src/http/ngx_http_request.c:1483
      #16 0x000055c216f4fb24 in ngx_http_process_request_line (rev=0x7f4f3c6fe200) at src/http/ngx_http_request.c:1154
      #17 0x000055c216f35996 in ngx_epoll_process_events (cycle=<optimized out>, timer=<optimized out>, flags=<optimized out>) at src/event/modules/ngx_epoll_module.c:901
      --Type <RET> for more, q to quit, c to continue without paging--z
      #18 0x000055c216f2b966 in ngx_process_events_and_timers (cycle=cycle@entry=0x55c219009950) at src/event/ngx_event.c:257
      #19 0x000055c216f34e68 in ngx_single_process_cycle (cycle=0x55c219009950) at src/os/unix/ngx_process_cycle.c:323
      #20 0x000055c216f09e6d in main (argc=<optimized out>, argv=<optimized out>) at src/core/nginx.c:382
      (gdb)
      

      APICAst config to replicate this:

      {
        "services": [
          {
            "id": 42,
            "backend_version": "1",
            "backend_authentication_type": "service_token",
            "backend_authentication_value": "token-value",
            "proxy": {
              "hosts": [
                "eloy",
                "one",
                "one.com"
              ],
              "api_backend": "https://echo-api.3scale.net",
              "proxy_rules": [
                {
                  "pattern": "/",
                  "http_method": "GET",
                  "metric_system_name": "hits",
                  "delta": 2
                },
                {
                  "pattern": "/",
                  "http_method": "POST",
                  "metric_system_name": "hits",
                  "delta": 2
                }
              ],
              "policy_chain": [
                {
                  "name": "apicast.policy.nginx_filters",
                  "configuration": {
                    "headers": [
                      {
                        "name": "If-Match",
                        "append": true
                      }
                    ]
                  }
                },
                {
                  "name": "content_caching",
                  "version": "builtin",
                  "configuration": {
                    "rules": [
                      {
                        "cache": true,
                        "header": "X-Cache-Status",
                        "condition": {
                          "combine_op": "and",
                          "operations": [
                            {
                              "left": "oo",
                              "op": "==",
                              "right": "oo"
                            }
                          ]
                        }
                      }
                    ]
                  }
                },
                {
                  "name": "apicast.policy.apicast"
                }
              ]
            }
          }
        ]
      }
      

      [0] https://github.com/3scale/APIcast/blob/master/t/apicast-policy-nginx_filters.t#L94-L148

              Unassigned Unassigned
              eloycoto Eloy Coto
              Jakub Urban Jakub Urban (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: