Welcome! Log In Create A New Profile

Advanced

Re: Optimize nginx for uploading/download huge files..

Maxim Dounin
January 05, 2012 08:06AM
Hello!

On Wed, Jan 04, 2012 at 07:38:08PM -0500, ronjpark wrote:

> stefancaunter Wrote:
> -------------------------------------------------------
> > On Tue, Jan 3, 2012 at 5:35 PM, ronjpark
> > <nginx-forum@nginx.us> wrote:
> >
> > The bottleneck is neither CPU nor memory. VM disk
> > performance will be
> > bad. Any throughput you get on the virtual disk
> > will degrade as each
> > concurrent upload client triggers reads and
> > writes. This is one case
> > where you need a SAN, ZFS filesystem, or an SSD
> > drive; configure with
> > large block size if files are >1GB. Even server
> > class hard drives will
> > thrash and slow down everything with enough
> > clients contending for the
> > disk. Remember, in relative time, if a cpu
> > operation takes a second,
> > an operation on the disk takes one month, under
> > ideal conditions, and
> > you are giving nginx less than ideal conditions.
> >
> > Stefan Caunter
> > http://scaleengine.com/contact
> >
>
> Thank you Stefan.
> I also doubted what you said and tested with 1MB file.
>
> client_body_buffer_size is set to 2M and proxy_buffer_size = 2M also.
> But it doesn't work.
>
> As my understainding, nginx doesn't make disk I/O for this file
> transaction because of enough big buffer size. Right?
> Please fix me if I'm wrong.

Yes.

To limit disk I/O for big requests/responses there are following
options available:

1. Using larger buffers, notably client_body_buffer_size,
proxy_buffer_size, proxy_buffers. This is basically what you've
already done.

2. For responses you may also completely disable disk buffering
using "proxy_max_temp_file_size 0". This implies that nginx won't
be able to read full response from a backend though, and
connection to a backend will be busy for a time needed to client to
download a response.

Directive description may be found here:
http://wiki.nginx.org/HttpProxyModule#proxy_max_temp_file_size

Maxim Dounin

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

Optimize nginx for uploading/download huge files..

ronjpark January 03, 2012 05:35PM

Re: Optimize nginx for uploading/download huge files..

stefancaunter January 04, 2012 06:34PM

Re: Optimize nginx for uploading/download huge files..

ronjpark January 04, 2012 07:38PM

Re: Optimize nginx for uploading/download huge files..

Maxim Dounin January 05, 2012 08:06AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 277
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready