Welcome! Log In Create A New Profile


Possible cached file corruption with aio_write enabled

May 25, 2016 08:15AM

I've recently upgraded one of the nginx servers within a caching (proxy_cache module) cluster from 1.8.1 to 1.10 and soon after I noticed unusually high number of various errors only on that server which I eventually pin pointed to a mismatch between the actual cached file size on disk and size reported in file metadata (either content-length or something else). Apparently, cached files on 1.10 are cut short and miss up to few hundred kilobytes at their ends (for files over 100 MB in total size). For a few files I checked, the larger file was, more content it missed at the end. Eventually I found the culprit to be aio_write which I enabled the same time I upgraded nginx. Disabling it and removing all already cached files resolved the problem.

Relevant directives:

thread_pool default threads=4 max_queue=65536;
sendfile on;
aio threads=default;
aio_write on;
# output_buffers are used if sendfile is not used
# output_buffers 8 512k;
read_ahead 1;

proxy_cache_path /cache ... use_temp_path=on;
proxy_buffering on;
proxy_buffers 32 64k;
proxy_busy_buffers_size 128k;
proxy_temp_file_write_size 128k;

OS is Linux 4.1.

Subject Author Posted

Possible cached file corruption with aio_write enabled

vedranf May 25, 2016 08:15AM

Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 76
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready