Just to complete the information, in my case I have two configuration according with the web server, so:

# TO VARNISH
location / {
proxy_redirect off; # Do not redirect this proxy - It needs to be pass-through
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Server-Address $server_addr;
proxy_connect_timeout 5s;
proxy_read_timeout 300s;
proxy_pass_header Set-Cookie;
proxy_pass http://localhost:6081; # Pass all traffic through to Varnish
}

TO APACHE
location / {
client_max_body_size 10m;
client_body_buffer_size 128k;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 32k;
proxy_buffers 8 16k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
proxy_connect_timeout 30s;
proxy_redirect http://domain.com:8081 http://domain.com;
proxy_redirect http://www.domain.com:8081 http://www.domain.com;

proxy_pass http://192.168.1.7:8081/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}

In the case varnish server not response, the request will forward to apache server.

Att,
Thiago Gomes