Just an update, I forgot to add,
It seems it's possibly related to chrome? (currently running Version 33.0.1750.117)
I converted the mp4 to ogv, same issue, but only in chrome, the android version of chrome refuses to load period. Firefox is fine.
So probably not nginx related at all.