Hi all,
I have installed nginx 0.8.35 (flv streaming and h264 module enabled) on my production video streaming server to see if it can perform better than lighttpd.
However, lighttpd outperforms nginx
+ speed
+ throughput
Here is my settings
+ CentOS release 5.3 (Final) with Lustre 1.8 patch
+ Kernel 2.6.18-92.1.17.el5_lustre.1.8.0smp x64
+ lighttpd-1.5.0
server.network-backend = "gthread-aio"
server.max-read-threads = 120
server.stat-cache-engine = "fam"
server.event-handler = "linux-sysepoll"
server.max-worker = 25
+ nginx 0.8.35
worker_processes 50; # I have tested with 4, 8, 20 processes
worker_connections 2048;
sendfile on;
tcp_nopush on;
tcp_nodelay off;
+ iptraf is used to monitor network activity in real time
All the video files is LustreFS based, network mounted. The real data is stored on 6 storage servers (Lustre OSS)
I have configured 2 1-GB network cards: one for outgoing data (streaming via port 80) and the another for Lustre client, reading data from storage servers
nginx
=====
When I turned nginx on, I saw strange things on network activity ( I used iptraf to see network activity per network cards)
+ Incoming data: data read from storage servers: 250 - 290 MBit/s (better than lighttpd - see below)
+ Outgoing data: data sent to clients (flash players, VLC ...): 90 - 140 Mbit/s (much worse than lighttpd - see below)
It means that nginx can read more data from disk than the throughput it can send to clients.
At 1300 - 1400 concurrent connections, it is expected to wait 42 - 51 s before my VLC player can create a new streaming connection.
lighttpd
=======
After that I turned off nginx and switched on lighttpd. The network activity seemed to be much more logical.
+ Incoming data: data read from storage servers: 180 - 220 MBit/s
+ Outgoing data: data sent to clients (flash players, VLC ...): 190 - 230 Mbit/s
It means that lighttpd can send everything it read from the disk to clients.
At 1300 - 1400 concurrent connections, it is expected to wait 16 - 23 s before my VLC player can create a new streaming connection. Much faster than in nginx.
I can not use sendfile on lighttpd because it is extremely slow.
Conclusion
==========
25 lighttpd worker processes outperforms 50 nginx processes in term of
+ response time
+ throughput
==========
What I really want to replace lighttpd with nginx is that lighttpd
+ crashed many time a day
+ caused very high load average.
+ unable to killall because it can become zombie after that. Every lighttpd processes switches to S status while streaming. nginx switches to D status.
But I still don't know why nginx is slower than lighttpd on sending data to clients while it can read more data from disk than the counterpart.
Any thought?
Regards,
Hoang