Wanted to follow up to say that you were spot on.

The file was ~800mb and it was indeed caching the whole thing. We could see the file get served to nginx and then nginx continued to serve to the client.

We were able to show that it was indeed doing least_conn by downloading a file that was multiple gigabytes in size. This exceeded the buffer and would tie up that connection. When that happened we would then see connections get routed to the other upstream.

Thanks so much for helping educate us.