Hi, I’m running nginx on Kubernetes and I want to use it as a UDP load balancer at high rate.
nginx.conf:
user root;
worker_processes auto;


events {
worker_connections 1024;
}

stream {
upstream app_node {
server <receiver_ip>:<receiver_port> fail_timeout=60s weight=1;
}

server {
listen <nginx_port> udp;
proxy_pass app_node;
proxy_protocol on;
set_real_ip_from 0.0.0.0/0;
proxy_bind $server_addr:$remote_port;
}
}

I try to test it by generating high traffic, such as 30,000 packets per second on average for 5 minutes, and each packet size is 1 KB.
I use tcpdump for monitoring packets that nginx receives and sends.

Tcpdump command for receiving packets:
tcpdump -i <receiving_inteface> dst <nginx_pod_ip> and port <nginx_port> -w receceived_30000.pcap

Tcpdump command for sending packets:
tcpdump -i <sending_inteface> src <nginx_pod_ip> and dst <receiver_ip> and port <receiver_port> -w sent_30000.pcap

But the count of packets captured by the two commands is not the same. It seems that nginx drops packets:

Result of first tcpdump command:
9000000 packets captured
9000000 packets received by filter
0 packets dropped by kernel

Result of second tcpdump command:
8997898 packets captured
8997898 packets received by filter
0 packets dropped by kernel

And this difference will increase if we increase the rate of sending packets.
So what is the problem here? Is our configuration wrong, or has something else happened?



Edited 1 time(s). Last edit at 09/30/2023 05:13AM by A_H.