504 Timeout Using to Many cUrl From Website That is Down

vezeCS asked:

I have a dedicated server with 32 gb ram / 8 processors.
I use on all my sites cUrl php displaying some code. The website from where I make the cUrl is a VPS where I use another cUrl with multiple proxies that switches every request.

So I use like this: on site A (Dedicated Server) curl to site B (VPS).
The cUrl code looks like this:

$url = "https://siteB.com/code";
$ch = curl_init();
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, 7);
curl_setopt ($ch, CURLOPT_TIMEOUT, 7);
$curl_res = curl_exec($ch);
curl_close ($ch);

My problem is that on a website (site A in this case) I have like 300k visitors per day and if the site B doesn’t respond for any reason the site A gets 504 timeout and the other websites on dedicated server works ok.

On error log from site A I have errors like this:

25329#25329: *3623 upstream timed out (110: Connection timed out) while reading response header from upstream

When I check Nginx I see that from 40-50 writing (that were usually) there are to many writing processes and from here it gets timeout

Active connections: 1756 
server accepts handled requests
11367 11367 11377 
Reading: 0 Writing: 1395 Waiting: 361

My nginx configuration is:

worker_processes auto;
worker_rlimit_nofile 65535;

events {
    multi_accept on;
    use epoll;
    worker_connections 65535;
}

http {
    ## Basic Settings ##
    client_body_buffer_size        128k;
    client_body_timeout            5s; # Use 5s for high-traffic sites
    client_header_timeout          5s; # Use 5s for high-traffic sites
    client_max_body_size           1024m;
    keepalive_timeout              30s;
    open_file_cache                max=200000 inactive=20s;
    open_file_cache_errors         on;
    open_file_cache_min_uses       2;
    open_file_cache_valid          30s;
    port_in_redirect               off;
    reset_timedout_connection      on;
    send_timeout                   5s;
    sendfile                       on;
    server_name_in_redirect        off;
    server_names_hash_bucket_size  1024;
    server_names_hash_max_size     1024;
    server_tokens                  off;
    tcp_nodelay                    on;
    tcp_nopush                     on;
    types_hash_max_size            2048;

    ## DNS Resolver ##
    # If in China, enable the OpenDNS entry that matches your network connectivity (IPv4 only or IPv4 & IPv6)
    # OpenDNS (IPv4 & IPv6)
    #resolver                      208.67.222.222 208.67.220.220 [2620:0:ccc::2] [2620:0:ccd::2];
    # OpenDNS (IPv4 only)
    #resolver                      208.67.222.222 208.67.220.220;
    # Google Public DNS (IPv4 & IPv6)
    #resolver                      8.8.8.8 8.8.4.4 [2001:4860:4860::8888] [2001:4860:4860::8844];
    # Google Public DNS (IPv4 only) [default]
    resolver                       8.8.8.8 8.8.4.4;

    ## Real IP Forwarding ##
    set_real_ip_from 127.0.0.1;

    # CloudFlare IPs
    # List from: https://www.cloudflare.com/ips-v4
    set_real_ip_from 103.21.244.0/22;
    set_real_ip_from 103.22.200.0/22;
    set_real_ip_from 103.31.4.0/22;
    set_real_ip_from 104.16.0.0/12;
    set_real_ip_from 108.162.192.0/18;
    set_real_ip_from 131.0.72.0/22;
    set_real_ip_from 141.101.64.0/18;
    set_real_ip_from 162.158.0.0/15;
    set_real_ip_from 172.64.0.0/13;
    set_real_ip_from 173.245.48.0/20;
    set_real_ip_from 188.114.96.0/20;
    set_real_ip_from 190.93.240.0/20;
    set_real_ip_from 197.234.240.0/22;
    set_real_ip_from 198.41.128.0/17;
    # List from: https://www.cloudflare.com/ips-v6
    set_real_ip_from 2400:cb00::/32;
    set_real_ip_from 2405:8100::/32;
    set_real_ip_from 2405:b500::/32;
    set_real_ip_from 2606:4700::/32;
    set_real_ip_from 2803:f800::/32;
    set_real_ip_from 2c0f:f248::/32;
    set_real_ip_from 2a06:98c0::/29;

    # Replace with correct visitor IP
    real_ip_header X-Forwarded-For;
    real_ip_recursive on;

    ## MIME ##
    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ## Logging Settings ##
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    ## Gzip Settings ##
    gzip on;
    gzip_buffers 16 8k;
    gzip_comp_level 5;
    gzip_disable "msie6";
    gzip_min_length 256;
    gzip_proxied any;
    gzip_types
        application/atom+xml
        application/javascript
        application/json
        application/ld+json
        application/manifest+json
        application/rss+xml
        application/vnd.geo+json
        application/vnd.ms-fontobject
        application/x-font-ttf
        application/x-javascript
        application/x-web-app-manifest+json
        application/xhtml+xml
        application/xml
        font/opentype
        image/bmp
        image/svg+xml
        image/x-icon
        text/cache-manifest
        text/css
        text/javascript
        text/plain
        text/vcard
        text/vnd.rim.location.xloc
        text/vtt
        text/x-component
        text/x-cross-domain-policy
        text/x-js
        text/xml;
    gzip_vary on;

    # Security Headers
    add_header        X-XSS-Protection "1; mode=block" always;
    add_header        X-Content-Type-Options "nosniff" always;

    # Info
    add_header        X-Nginx-Cache-Status $upstream_cache_status;
    add_header        X-Server-Powered-By "Engintron";

    # Proxy Cache Settings
    proxy_cache_path  /var/cache/nginx/engintron_dynamic levels=1:2 keys_zone=engintron_dynamic:20m inactive=10m max_size=512m;
    proxy_cache_path  /var/cache/nginx/engintron_static levels=1:2 keys_zone=engintron_static:20m inactive=10m max_size=512m;
    proxy_temp_path   /var/cache/nginx/engintron_temp;

    ## Virtual Host Configs ##
    include /etc/nginx/conf.d/*.conf;
}

What I ask is, how can I make the cUrl check if the website B is slow or down to stop that request if no response in 4-5-6 seconds? Or there is something I can increase in my configuration? I also see that Load Averages doesn’t go more than 1.5 (this was the maximum for all the server).

System Status now:

Processors: 8 CPU(s)

RAM: 31.19 GB

Memory Usage:
              total        used        free      shared  buff/cache   available
Mem:          31940        2155        2301         686       27484       28449
Swap:          1021          22         999

From what I see there is a limit somewhere that if the X connections is reached it gets slowed or timeout, but don’t know were.

On my Dedicate Server I have all setup configuration to default from WHM Cpanel, except the configuration from nginx.

P.S. I’m not good at all with servers and if can someone guide me.

My answer:


You set send_timeout (and several other timeouts) to 5 seconds, but your PHP code that calls the external web site waits up to 7 seconds for it to respond. Obviously this won’t work. You need to change one or both.


View the full question and any other answers on Server Fault.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.