Роман, Дмитрий, спасибо за рекомендации.

Размер кеша уменьшил до 10 Гб, включил дебаг.

Проблема пока не проявляется, но в дебаг логе вижу подозрительные вещи:

2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #1 1 29fddad7
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #0 1 35fdec05
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #1 1 29fddad7
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #0 1 39769558
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #1 1 29fddad7
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #0 1 1ef351b0
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #1 1 29fddad7
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #0 1 0b4eaa2e
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #1 1 29fddad7
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #0 1 81d7ae20
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #1 1 29fddad7
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #0 1 d80ef979
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #1 1 29fddad7
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #0 1 e7875a78
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #1 1 29fddad7
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #0 1 c2ad99cc
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #1 1 29fddad7
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #0 1 6f3d1738
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #1 1 29fddad7
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #0 1 bc7b6d5a
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #1 1 29fddad7
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #0 1 9bee7f9c
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #1 1 29fddad7
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #0 1 b10ba9bf
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #1 1 29fddad7
2015/04/28 13:57:07 [debug] 22500#0: http file cache forced expire: #0 1 68f86aaf

(Обратите внимание на 29fddad7)

При этом в файловой системе я его не нахожу (в удаленных, но незкакрытых файлах тоже нет):

# ls -all /opt/nginx-cache-images1/29/fd/
total 96
drwx------ 2 www-data www-data 12288 Apr 28 14:00 .
drwx------ 258 www-data www-data 4096 Apr 17 11:04 ..
-rw------- 1 www-data www-data 11120 Apr 26 22:08 187415d376209c37038d3e536a4cfd29
-rw------- 1 www-data www-data 4667 Apr 28 13:54 77b0dc15b1aec3ce8bb1d3d904ccfd29
-rw------- 1 www-data www-data 14241 Apr 28 13:57 85e329c59da26ddac0e5844adc23fd29
-rw------- 1 www-data www-data 17182 Apr 28 13:56 90d0b75025acf141f8fd5f0bc431fd29
-rw------- 1 www-data www-data 5297 Apr 28 13:52 9d1edeca10f5f596d1bb13d62e7dfd29
-rw------- 1 www-data www-data 13903 Apr 28 13:56 d7deedc7100d675b024eaebb99c6fd29