I figured out what it was. I had an error_page directive in another location block in the same server.conf that was apparently overriding the proxy_next_upstream. I commented it out and now the upstream throwing the 404 is being skipped. I'm just going to remove 404 from the error_page directive.