Dear Maxim,

thank you for your suggestions. Analyzing the code the following turned out about http_write_filter:
1. it is able to buffer output (eg. postpone_output)
2. can delay response before sending bytes (limit <= 0)
3. delays response after sending bytes (nsent - sent)

As you mentioned we delay sending the last byte of the response and only do millisecond calculation when sending it.

$ cat limit_rate.patch
--- src/http/ngx_http_write_filter_module.c 2012-01-18 16:07:43.000000000 +0100
+++ src/http/ngx_http_write_filter_module.c 2012-08-22 12:44:03.862873715 +0200
@@ -47,9 +47,10 @@
ngx_int_t
ngx_http_write_filter(ngx_http_request_t *r, ngx_chain_t *in)
{
- off_t size, sent, nsent, limit;
+ off_t size, sent, nsent, limit, nlimit;
ngx_uint_t last, flush;
ngx_msec_t delay;
+ ngx_time_t *t;
ngx_chain_t *cl, *ln, **ll, *chain;
ngx_connection_t *c;
ngx_http_core_loc_conf_t *clcf;
@@ -214,6 +215,23 @@
limit = r->limit_rate * (ngx_time() - r->start_sec + 1)
- (c->sent - clcf->limit_rate_after);

+ if (last && size == 1) {
+ t = ngx_timeofday();
+
+ if (t->msec < r->start_msec) {
+ t->sec--;
+ t->msec += 1000;
+ }
+
+ nlimit = r->limit_rate * (t->sec - r->start_sec)
+ + r->limit_rate * (t->msec - r->start_msec) / 1000
+ - (c->sent + size - clcf->limit_rate_after);
+
+ if (nlimit <= 0) {
+ limit = nlimit;
+ }
+ }
+
if (limit <= 0) {
c->write->delayed = 1;
ngx_add_timer(c->write,
@@ -224,6 +242,12 @@
return NGX_AGAIN;
}

+ if (last && limit > size - 1) {
+ if (size > 1) {
+ limit = size - 1;
+ }
+ }
+
if (clcf->sendfile_max_chunk
&& (off_t) clcf->sendfile_max_chunk < limit)
{

It also turned out that setting sendfile_max_chunk to a small enough value is also a solution for our problem, but this patch also works with default sendfile_max_chunk = 0 setting.

Anyway, in nginx 1.2.3 source we found this:
if (size == 0 && !(c->buffered & NGX_LOWLEVEL_BUFFERED)) {
if (last) {
r->out = NULL;
c->buffered &= ~NGX_HTTP_WRITE_BUFFERED;

return NGX_OK;
}

if (flush) {
do {
r->out = r->out->next;
} while (r->out);

c->buffered &= ~NGX_HTTP_WRITE_BUFFERED;

return NGX_OK;
}

Instead we could use this if I am right:
if (size == 0 && !(c->buffered & NGX_LOWLEVEL_BUFFERED)) {
if (last || flush) {
r->out = NULL;
c->buffered &= ~NGX_HTTP_WRITE_BUFFERED;

return NGX_OK;
}

Thanks for your help.

Regards,
Gabor