Ahh right, so basically I have to copy:

valid_referers server_names not-my-domain.com;
if ($invalid_referer) {
return 403;
}

Into the php match block. Is there a way to do this without having the same exact code copied into both location blocks?