Nginx post-ssl-termination debug mirror

Michael Altfield asked:

How can I tell nginx to output all incoming traffic for a given server{} to a specific/additional/redundant location after decrypting it (like port mirroring on routers) so that it can be analyzed for troubleshooting purposes?

I need to debug the traffic coming-in to an https-only website. The server is headless, and I’d rather

  1. Not install a GUI just for wireshark
  2. Not move my private keys off the server for later analysis on my local GUI with wireshark

So wireshark (the most commonly-cited solution for decrypting an https stream) is not a solution. I want something that’s CLI-only and requires minimal tools done live on the server. I checked tcpdump‘s https support; it’s absent but you can combine a ton of other tools and get it functional, but it’s not exactly ideal.

My web server is nginx. I’m wondering if there’s some way to setup nginx such that–in addition to the way it normally handles its traffic for a given server{} block–send the decrypted traffic to some mirror (just for debugging) that I can then analyze directly with tcpdump, ideally in real-time.

Is it possible to add a proxy_pass mirror to an nginx server{} block such that nginx handles the traffic as it usually does plus sending the decrypted traffic to some "debug" location (which could just be an nc -l 8080 or something)?

My answer:

I found out recently that nginx does indeed have this functionality beginning with version 1.13.4. Using the mirror directive you can forward a copy of each request to a different HTTP server. The response, if any, is ignored.

In the server or location for which you want requests mirrored, add a mirror directive with a URL path that will never occur on your web site. Then, add a new location that serves exactly that path and passes the requests to some other server.

A simplistic example:

server {
    location / {
        mirror /Jahg3bioshes;
        # ... the rest of your directives

    location = /Jahg3bioshes {
        proxy_pass http://sink.server.example$request_uri;

Note that while nginx ignores the response, it does not ignore the time taken to serve the response, and if the mirror server is slow to respond, it will also slow down the original request. It should probably just be some trivial server, e.g. a freshly installed web server with the distribution default configuration.

View the full question and any other answers on Server Fault.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.