I have a wordpress multisite setup. So I looked around and found the "correct" way to set up the location directive. Apparently this should work according to the nginx codex example (http://codex.wordpress.org/Nginx)

location ~ ^/[_0-9a-zA-Z-]+/files/(.*)$ {
try_files /wp-content/blogs.dir/$blogid/files/$2 /wp-includes/ms-files.php?file=$2 ;
access_log off; log_not_found off; expires max;
}

If I try to access domain.com/files/test.css, I would expect to see the file in /wp-content/blogs.dir/$blogid/files/test.css but I get a 404. (Btw, shouldn't the backreference be $1?) The only way I can get it to almost work is like this:

location ^~/files/ {
try_files /wp-content/blogs.dir/$blogid/$uri /wp-includes/ms-files.php?file=$2;
}

But then I dont get the file in $2 to be used in the fallback to ms-files.php. And now this is very strange, I expected this to work:

location ^~/files/(.*)$ {
try_files /wp-content/blogs.dir/$blogid/$uri /wp-includes/ms-files.php?file=$1;
}

But then I get the 404 again so It stops working completely! It's like the location isn't matched anymore, at least /wp-content/blogs.dir/$blogid/$uri should work right?

Also tried location ~^/files/(.*)$ and a few other variants, what am I missing here? I have looked around and this should work according to lots of examples I found.

The $blogid variable is correct, I have set up the map using nginx-helper plugin.