Dear Maxim,

thank you for your suggestions. Analyzing the code the following turned out about http_write_filter:

1. it is able to buffer output (eg. postpone_output)

2. can delay response before sending bytes (limit <= 0)

3. delays response after sending bytes (nsent - sent)

As you mentioned we delay sending the last byte of the response and only do millisecond calculation when sending it.

$ cat limit_rate.patch

--- src/http/ngx_http_write_filter_module.c 2012-01-18 16:07:43.000000000 +0100

+++ src/http/ngx_http_write_filter_module.c 2012-08-22 12:44:03.862873715 +0200

@@ -47,9 +47,10 @@

ngx_int_t

ngx_http_write_filter(ngx_http_request_t *r, ngx_chain_t *in)

{

- off_t size, sent, nsent, limit;

+ off_t size, sent, nsent, limit, nlimit;

ngx_uint_t last, flush;

ngx_msec_t delay;

+ ngx_time_t *t;

ngx_chain_t *cl, *ln, **ll, *chain;

ngx_connection_t *c;

ngx_http_core_loc_conf_t *clcf;

@@ -214,6 +215,23 @@

limit = r->limit_rate * (ngx_time() - r->start_sec + 1)

- (c->sent - clcf->limit_rate_after);

+ if (last && size == 1) {

+ t = ngx_timeofday();

+

+ if (t->msec < r->start_msec) {

+ t->sec--;

+ t->msec += 1000;

+ }

+

+ nlimit = r->limit_rate * (t->sec - r->start_sec)

+ + r->limit_rate * (t->msec - r->start_msec) / 1000

+ - (c->sent + size - clcf->limit_rate_after);

+

+ if (nlimit <= 0) {

+ limit = nlimit;

+ }

+ }

+

if (limit <= 0) {

c->write->delayed = 1;

ngx_add_timer(c->write,

@@ -224,6 +242,12 @@

return NGX_AGAIN;

}

+ if (last && limit > size - 1) {

+ if (size > 1) {

+ limit = size - 1;

+ }

+ }

+

if (clcf->sendfile_max_chunk

&& (off_t) clcf->sendfile_max_chunk < limit)

{

It also turned out that setting sendfile_max_chunk to a small enough value is also a solution for our problem, but this patch also works with default sendfile_max_chunk = 0 setting.

Anyway, in nginx 1.2.3 source we found this:

if (size == 0 && !(c->buffered & NGX_LOWLEVEL_BUFFERED)) {

if (last) {

r->out = NULL;

c->buffered &= ~NGX_HTTP_WRITE_BUFFERED;

return NGX_OK;

}

if (flush) {

do {

r->out = r->out->next;

} while (r->out);

c->buffered &= ~NGX_HTTP_WRITE_BUFFERED;

return NGX_OK;

}

Instead we could use this if I am right:

if (size == 0 && !(c->buffered & NGX_LOWLEVEL_BUFFERED)) {

if (last || flush) {

r->out = NULL;

c->buffered &= ~NGX_HTTP_WRITE_BUFFERED;

return NGX_OK;

}

Thanks for your help.

Regards,

Gabor