Ok, some updates on the testing. Using the configuration listed below does allow me to display index.php, but not index.html:
http {
...
upstream backend {
server 192.168.0.2:8000;
server 192.168.0.3:8000;
server 192.168.0.4:8000;
}

upstream fastcgi {
server 192.168.0.2:9000;
server 192.168.0.3:9000;
server 192.168.0.4:9000;
}
...
}

server {
listen publicip:80 default_server backlog=256 rcvbuf=32k sndbuf=8k;
server_name domain.com;
root /var/www/html;
index index.php index.html;

location / {
proxy_pass http://backend;
}

location ~ \.php$ {
fastcgi_pass fastcgi;
include fastcgi.conf;
}
}

I tried to use only one server as main entrance but the load is going very high, once I run siege on it.
How would I enable support for regular html files in the above configuration?

Thanks.