Hi all,

I have installed nginx 0.8.35 (flv streaming and h264 module enabled) on my production video streaming server to see if it can perform better than lighttpd.

However, lighttpd outperforms nginx

+ speed
+ throughput

Here is my settings

+ CentOS release 5.3 (Final) with Lustre 1.8 patch
+ Kernel 2.6.18-92.1.17.el5_lustre.1.8.0smp x64
+ lighttpd-1.5.0

server.network-backend = "gthread-aio"
server.max-read-threads = 120
server.stat-cache-engine = "fam"
server.event-handler = "linux-sysepoll"
server.max-worker = 25

+ nginx 0.8.35

worker_processes 50; # I have tested with 4, 8, 20 processes
worker_connections 2048;
sendfile on;
tcp_nopush on;
tcp_nodelay off;

+ iptraf is used to monitor network activity in real time

All the video files is LustreFS based, network mounted. The real data is stored on 6 storage servers (Lustre OSS)

I have configured 2 1-GB network cards: one for outgoing data (streaming via port 80) and the another for Lustre client, reading data from storage servers

nginx
=====

When I turned nginx on, I saw strange things on network activity ( I used iptraf to see network activity per network cards)

+ Incoming data: data read from storage servers: 250 - 290 MBit/s (better than lighttpd - see below)
+ Outgoing data: data sent to clients (flash players, VLC ...): 90 - 140 Mbit/s (much worse than lighttpd - see below)

It means that nginx can read more data from disk than the throughput it can send to clients.

At 1300 - 1400 concurrent connections, it is expected to wait 42 - 51 s before my VLC player can create a new streaming connection.

lighttpd
=======

After that I turned off nginx and switched on lighttpd. The network activity seemed to be much more logical.

+ Incoming data: data read from storage servers: 180 - 220 MBit/s
+ Outgoing data: data sent to clients (flash players, VLC ...): 190 - 230 Mbit/s

It means that lighttpd can send everything it read from the disk to clients.

At 1300 - 1400 concurrent connections, it is expected to wait 16 - 23 s before my VLC player can create a new streaming connection. Much faster than in nginx.

I can not use sendfile on lighttpd because it is extremely slow.

Conclusion
==========
25 lighttpd worker processes outperforms 50 nginx processes in term of

+ response time
+ throughput

==========

What I really want to replace lighttpd with nginx is that lighttpd
+ crashed many time a day
+ caused very high load average.
+ unable to killall because it can become zombie after that. Every lighttpd processes switches to S status while streaming. nginx switches to D status.

But I still don't know why nginx is slower than lighttpd on sending data to clients while it can read more data from disk than the counterpart.

Any thought?

Regards,

Hoang