Thank you all for your replies.

Since all 3 replies suggest some form of caching I'll respond to them together here -
The nginx servers that I mentioned in my post do not serve client requests directly, the clients always hit the CDN first (we use mostly Akamai), and the CDN then pulls from these nginx servers. In other words, these servers act as the CDN origin. Therefore, hot / popular content is already taken care of - I have no problem there.
Since the files we serve are large (video) the CDN isn't caching them for too long (we send caching header of 3 months, and the files usually get cached for a couple of days), so the servers are getting quite a few requests, and these requests hardly repeat themselves. Each server is delivering roughly 1/2TB of data per day, so to get any hits on an NFS cache we'll probably need a very large cache. And even if do that, we'll still have this problem with the non-popular content (e.g. videos that are watched on average once a week) - such a request may hang the process if opening the file takes a long time.

Thanks,

Eran