Hi!

I'm not a nginx developer and I've not even seen the source yet, however I've nginx running on cache and I can reply by experience one of your questions.

As long as you have free memory and your os (mine is debian too) correctly uses that memory to cache, you won't see a single IO Read into the cache files by nginx. You will see the first io read to the cached files whenever you have no more free memory to cache.

Please share later your experience :)

In my case in case it helps someone else, I had serious issues with IO but didn't blame nginx but my current virtualization setup: XenServer from Citrix over two SATA disks using software raid-1, using lvm for partitions and having on the same host other vm's with the backends, mysql database, etc. My IO was killing me when reaching a few hundreds requests per second, leaving sometimes most of workers locked on IO wait. After tunning and tunning nginx decided it wasn't the one to blame and fixed it other way: added SSD disk (software RAID-1 and LVM too) to the host and mounted a partition on the nginx debian machine exclusively for nginx cache, temp folder and logs. Since then i've not been able to throw enough load to have a IO problem again :-).

My current iostats on the production server:
avg-cpu:
%user %nice %system %iowait %steal %idle
0.10 0.03 0.31 0.39 0.02 99.15

Cheers

Guzman