+1 to the idea.

Maybe something like:
limit_req one burst=10 nodelay=5; # first 5 'bursts' don't have a delay, the next 5 do

I haven't tried, but I suspect this doesn't do the desired thing:
limit_req one burst=10;
limit_req one burst=5 nodelay;

(I'm guessing that the first directive above essentially overrides the second for the first 5, then the second directive overrides the first after that)