nginx deny all requests to a virtual host . requests are coming from a nginx tcp forwarder

Laksith asked:

Summary – need to add whitelist ips to the mysite1.example.com. now when they are added it doesn’t work as every request is coming as originated from the load balancer server.

Im working on a setup with a front seating Nginx host with a upstream backend to loadbalance all tcp packets on port 443 to backend servers.

nginx config of Loadbalancer server running nginx – server C is as below

stream {
        upstream stream_backend {
                hash $remote_addr consistent;
                server 10.15.15.3:443;   ## server A
                server 10.15.15.9:443;   ## server B
        }


        server {
                listen     443;
                proxy_pass stream_backend;
                proxy_timeout 5s;
                proxy_connect_timeout 5s;
        }
}

server A and server B has below nginx.conf. they are identical servers with apps.

it has two virtual hosts running in each. they are working fine.

http {

    server {
        server_name mysite1.example.com;
        listen *:443 ssl;
        listen [::]:443 ssl;
        
        allow 123.45.85.220; # this seems not working
        deny all; # only this is working

        
        location ^~ /static/ {
            ...
        }
        ...
        
        ssl_certificate        file.pem;
        ssl_certificate_key    file.key;
    }


    server {
        server_name mysite2.example.com;
        listen *:443 ssl;
        listen [::]:443 ssl;
        
        
        location /somethin {
            ...
        }
        location /something2{
            ...
        }
        
        ssl_certificate        file.pem;
        ssl_certificate_key    file.key;
    }
}

what I need is to whitelist only few ips to the virtual host mysite1.example.com.
the issue I face is that the nginx running on Server A and B see the load balancer Ip as the client Ip. so when tried adding allow IP; deny all. doesn’t work for any host as it has the load balancer IP on all requests as the client IP.

Can someone guide me on adding proxy IP configs to achieve the above mentioned setup running fine.
Setup is complete except for the IP whitelist issue.

p.s SSl termination happens at the back-end servers , server A and Server B

I’ve searched through the web and found these helpful but still couldn’t figure out how to get it all working.

https://stackoverflow.com/questions/40873393/nginx-real-client-ip-to-tcp-stream-backend
https://www.cyberciti.biz/faq/nginx-redirect-backend-traffic-based-upon-client-ip-address/

My answer:


This is happening because your nginx load balancer makes a new TCP connection to your nginx web servers, causing the original client IP address to be lost.

You can solve this using the PROXY protocol. This protocol sends the original IP address information when the new connection is opened so that your web servers can be aware of it.

To set it up, you need to make the following changes:

  • On the load balancer, set proxy_protocol on;:

    stream {
        server {
            proxy_pass somewhere;
            proxy_protocol on;
    
  • On the web server, accept the PROXY protocol, and enable the Real IP functionality for the load balancer’s IP address.

    server {
        listen 443 ssl http2 proxy_protocol;
        listen [::]:443 ssl http2 proxy_protocol;
        real_ip_header proxy_protocol;
        set_real_ip_from <load balancer's IP>;
    

There are additional tweaks you can make to the PROXY protocol which you can find in the nginx documentation but this should get you started and solve the immediate problem.


View the full question and any other answers on Server Fault.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.