To elaborate a bit more, in a single location I may end up with something like this...

set $Whitelisted "No";
if ($GeoList1 = allow) {
set $Whitelisted "Yes";
}
if ($GeoList5 = allow) {
set $Whitelisted "Yes";
}
if ($http_user_agent ~ (googlebot|bingbot)) {
set $Whitelisted "Yes";
}
if ($Whitelisted ~* "No") {
limit_conn conlimit-one 5;
limit_req zone=limit-half burst=9;
}

So that I can allow both IPs and user agents to avoid getting limited.