Finally got it working.
Changed my setup such that Apache only listened on internal IP 127.0.0.1 and moved the auth process to Nginx.
I think (but didn't verify) the behaviour I saw may have been because my htpasswd file had been created using only the -c flag and not -c -d.