Hello!

On Fri, Sep 30, 2011 at 12:24:13PM +0300, Anatoli Marinov wrote:

> There found 2 solutions for this:
> 1. In ngx_http_file_cache_update the call of
> ngx_shmtx_lock(&cache->shpool->mutex); may be moved before
> rc = ngx_ext_rename_file(&tf->file.name, &c->file.name, &ext);
> I should work but there will be another syscall when the resource is
> locked.

This is really bad, as file rename may take a while. Especially
in case of temp path and cache path residing on different
filesystems.

Addtionally, I suspect this wouldn't be sufficient in case of
open_file_cache used.

> 2. When cache file is read from cache directory its binary header
> may be compared with shared memory structure and if shared mem
> structure is different it should replaced with the last one read
> from the file.

This is probably a way to go.

Maxim Dounin

>
> I have patched the approach 1 and I am starting the automated test now.
>
>
> On 09/30/2011 12:15 PM, Maxim Dounin wrote:
> >Hello!
> >
> >On Thu, Sep 29, 2011 at 01:00:20PM +0300, Anatoli Marinov wrote:
> >
> >>I invested more time to investigate the issue and I found something.
> >>
> >>Let's assume there is 2 nginx workers with pids 6362 and 6364 (they
> >>are actually 4 but only these 2 are interested).
> >>There are 3 parallel requests 10102, 10188 and 10232 for one and the
> >>same file.
> >>
> >>See the scenario. It describes the issue:
> >>
> >>--- context switch to 6362---
> >>6362 - 10102 - begin (connection accepted)
> >>6362 - 10102 - file do not exist in the cache
> >>6362 - 10102 - upstream peer found
> >>
> >>--- context switch to 6364 ---
> >>6364 - 10188 - begin (connection accepted)
> >>6364 - 10188 - file do not exist in the cache
> >>6364 - 10188 - upstream peer found
> >>6364 - 10188 - header received (header size 369 bytes)
> >>6364 - 10188 - file received
> >>6364 - 10188 - file renamed (with header size 369 bytes)
> >>6364 - 10188 - shared mem updated - header size 369 bytes
> >>6364 - 10188 - success end
> >>
> >>--- context switch to 6362 ---
> >>6362 - 10102 - header received (header size 373 bytes)
> >>6362 - 10102 - file received
> >>6362 - 10102 - file renamed (with header size 373 bytes)
> >> shared memory is not updated because the cpu context is switched
> >>or the process works in parallel on different cpu
> >>
> >>--- context switch to 6364 ---
> >>6364 - 10232 - begin (connection accepted)
> >>6364 - 10232 - record is found in shared memory - file exists
> >>(header size is 369 from shared mem)
> >>6364 - 10232 - header is read from the file (but only first 369 bytes)
> >>
> >>--- context switch to 6362 ---
> >>6362 - 10102 - shared mem updated - header size 373 bytes
> >>....
> >>....
> >>
> >>--- context switch to 6364 ---
> >>6364 - 10232 - http header could not be parsed because there are
> >>missing 4 bytes
> >>6364 - 10232 - failed end
> >>
> >>See the attached file. The scenario it taken from it.
> >>In general it is very hard for reproduction.
> >>I hit is 2-3 times for 10 000 000 requests. But the issue exists :).
> >Thanks for investigating this. I'll take a look how to fix this
> >properly.
> >
> >Maxim Dounin
> >
> >_______________________________________________
> >nginx-devel mailing list
> >nginx-devel@nginx.org
> >http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel

_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel