Welcome! Log In Create A New Profile

Advanced

1.6.2: Possible reasons for nginx stops responding after file access io error.

Posted by kabalin 
1.6.2: Possible reasons for nginx stops responding after file access io error.
November 21, 2016 07:07AM
I know, 1.6. is no longer maintained and the general advice would be to upgrade (which has been scheduled following the incident), but 1.6.2 is the version that Debian stable provides. Anyway, any thoughts on the possible reasons why Nginx could stop responding completely for exactly 30 minutes following the error below?

2016/11/07 16:38:57 [crit] 9857#0: *6789258 pread() "/srv/data/filedir/07/d7/07d760d40a089ac7cde7c6805e48982889820ef8" failed (5: Input/output error) while sending response to client....

And then the next record in the access log dated exactly 30 mins later (on a busy server that handles circa 300 simulations connections). Nothing in the syslog. CPU use was high during outage, but there is no indication that any other processes was using it. /srv/data is NFS mount (no errors related to NFS either, but systems team reported some volume access slowness on filestore end during that period). File reported in the error is quite small (~3mb). Issue occurred on two Nginx frontends at the few seconds difference (same version, same error, same outage duration, different file on the same datastore).

Is it something typical someone might knew or experienced? If not, do not bother really, I will keep an eye for behaviour in backported 1.9.10 we upgraded to at the moment and will come up with more details or bug report if we hit the issue again. I am quite happy with Nginx performance and reliability, first issue in last 5 years really.

Thanks,
Ruslan
Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 63
Record Number of Users: 8 on December 15, 2016
Record Number of Guests: 386 on August 02, 2016
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready