Hi Maxim,

Sorry for my late reply.

We use Nginx as CDN Cache. And it is normal case that the responses are
different from different origin servers. For instance: the origin server
could be
the customer server or some other CDN provider Cache server. The response
header could be different.

In race condition, I found the inconsistency between disk file and metadata
in
shared memory.
When (c->uniq != of.uniq) happens, I saw " too long header" in error log.

To fix this, I picked the ngx_pagesize to initiate c->body_start again.
then the "too long header"
is gone from error log.

I will try your patches next week. Thanks!


On Wed, Nov 28, 2018 at 4:45 AM Maxim Dounin <mdounin@mdounin.ru> wrote:

> Hello!
>
> On Mon, Nov 26, 2018 at 06:11:33PM -0800, Yongguang Bai wrote:
>
> > # HG changeset patch
> > # User Yongguang Bai <yongguang.bai@quantil.com>
> > # Date 1543276278 28800
> > # Mon Nov 26 15:51:18 2018 -0800
> > # Node ID 4d48e1f3ebf6f0634e8e2cbd057d9fcfbea9d988
> > # Parent a7ff19afbb14795fef14f599a304d0ad21052b70
> > Mail: nginx_error:cache file has too long header
> >
> > This error is printed when cached file is expired and the
> > response is changed in proxied server, and Nginx is under
> > heavy load.
> >
> > diff -r a7ff19afbb14 -r 4d48e1f3ebf6 src/http/ngx_http_file_cache.c
> > --- a/src/http/ngx_http_file_cache.c Mon Nov 26 18:29:56 2018 +0300
> > +++ b/src/http/ngx_http_file_cache.c Mon Nov 26 15:51:18 2018 -0800
> > @@ -376,6 +376,10 @@
> > ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
> > "http file cache fd: %d", of.fd);
> >
> > + if(c->uniq != of.uniq){
> > + c->body_start = ngx_pagesize;
> > + }
> > +
> > c->file.fd = of.fd;
> > c->file.log = r->connection->log;
> > c->uniq = of.uniq;
>
> This change looks wrong. In particular, ngx_pagesize, which is
> usually 4k, can be smaller than existing c->body_start, and this
> change will make things worse instead of fixing anything.
>
> Also, could you please elaborate what exactly you are trying to
> fix?
>
> There is a known race which may lead to "cache file has too long
> header" error - and, actually the error was added to ensure that
> this race is properly handled if happens, see here:
>
> http://hg.nginx.org/nginx/rev/6f97afc238de
> http://mailman.nginx.org/pipermail/nginx-devel/2011-September/001287.html
>
> Note that this race can only result in problems if for some reason
> the backend returns responses with different headers nearly at the
> same time. This may indicate that there is something wrong with
> the backend.
>
> Also, there is a known bug with multiple variants which also can
> lead to the same error. The patch can be found here:
>
> http://mailman.nginx.org/pipermail/nginx-devel/2018-January/010774.html
>
> If you are hitting this bug, please test if the patch works for
> you.
>
> --
> Maxim Dounin
> http://mdounin.ru/
>


--

Best Regards,

*Yongguang Bai*

Engineering Manager
[image: Quantil]

Connecting users with content...it's that simple.

Direct: +1-408-421-6265
Office: +1-888-847-9851

[image: Tweeter] https://twitter.com/Team_Quantil [image: Google Plus]
https://plus.google.com/+Quantil_team/ [image: Linked In]
https://www.linkedin.com/company/quantil
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel