I've had 2 cases with status code 500 now since setting error log to debug level:

The error msg: "epoll_wait() reported that client prematurely closed connection while sending request to upstream"

It's interesting to note that:
If a "normal" file (no caching involved) is requested and the client closes the connection prematurely, the status code is 200 and the response body is 0 bytes.
If first a php script is called, which responds with a X-Accel-Redirect to the cached file, and the client closes the connection prematurely, the status code is 500 and the response body is 0 bytes.