Thanks for the hint! I haven't worked with domain names in proxy_pass before and was sure that they're resolved dynamically. I changed the config to:

set $upstream_var example.com;
proxy_pass https://$upstream_var;