Finally got it working.

Changed my setup such that Apache only listened on internal IP 127.0.0.1 and moved the auth process to Nginx.

I think (but didn't verify) the behaviour I saw may have been because my htpasswd file had been created using only the -c flag and not -c -d.