Am 18.08.2021 um 03:14 schrieb Robert Mueller:
>> First, thanks for the patch.
>>
>> While the reuseport could cure (or hide if you will) the unbalancing you
>> see it makes sense to get better understanding what exactly is going on.
>> So far we haven't seen such weird behaviour ourself neither received
>> reports about such uneven connections distribution among nginx workers.
>>
>> Any chances you have accept_mutex and/or multi_accept? Any other ideas?
> xxxx
>
> As you can see, without the reuseport option, this causes severe scalability problems for us.

Hi,

I just checked, we see the same problem, the connection distribution is
very uneven. One process is using 30% CPU, one 15% CPU, and the rest is
below 5%.

# for i in 1285 1286 1287 1288 1289 1290 1291 1292; do echo "$i - " `ls
/proc/$i/fd | wc -l`; done
1285 -  106
1286 -  9447
1287 -  430
1288 -  2222
1289 -  76
1290 -  48
1291 -  24447
1292 -  42

We are using
nginx version: nginx/1.20.1

Configuration snippet:
```
worker_processes auto;
worker_rlimit_nofile 120000;

events {
    worker_connections 60000;
}

http {
...
}

mail {
  auth_http  127.0.0.1/mailauthXXXXX;

  proxy     on;
  starttls  on; ## enable STARTTLS for all mail servers

  ssl_prefer_server_ciphers  on;
  ssl_protocols              TLSv1.3 TLSv1.2 TLSv1.1 TLSv1;
  ssl_ciphers                xxxx;
  ssl_session_cache          shared:TLSSL:16m;
  ssl_session_timeout        10m;

  imap_auth login plain;
  ## STARTTLS is appended because of starttls directive above,
AUTH=LOGIN and AUTH=PLAIN are also appended automatically
  imap_capabilities  "IMAP4rev1" "LITERAL+" "SASL-IR" "LOGIN-REFERRALS"
"ID" "ENABLE" "IDLE" "NAMESPACE";
  pop3_capabilities  "TOP" "USER" "UIDL";

  server {
    listen                   a.b.c.d:993 ssl;
    listen                   [xxx:xxx:xxx:xxx::xxx]:993 ssl;
    protocol                 imap;
    server_name              imap.domain.com;
    auth_http_header         X-Auth-Port  993;
    auth_http_header         X-Domain     "imap.domain.com";
    auth_http_header         X-Auth       "xxxx";
    auth_http_header         User-Agent   "Nginx POP3/IMAP4 proxy";
    proxy_pass_error_message on;
    ssl_certificate          xxx.crt;
    ssl_certificate_key      xxx.key;
  }
}
```

Just wanted to show that you are not alone, Rob :-)

Michael

_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel