Welcome! Log In Create A New Profile

Advanced

Re: Parallel subrequests for multi-source long polling?

November 19, 2009 03:54AM
agentzh Wrote:
-------------------------------------------------------
> On Wed, Nov 18, 2009 at 4:24 PM, agentzh wrote:
> > 4. In your output filter, first check if there's
> a ctx object
> > associated with the current request. If yes,
> then you're filtering one
> > of the subrequests that your content handler
> starts. So now simply
> > buffer the response chain link in this ctx
> object (in meomory or on
> > disk, it's up to you).
>
> We should also abort the current (sub)request if
> the ctx shows
> "success", which means another brother subrequest
> has succeeded
> already ;)
>
> > If the "failures" counter reaches the total
> number of the subrequests
> > (which means all tests are failed), then returns
> the final failure
> > (and set the main request's status code if
> appropriate and send
> > headers for the main request.)
> >
> > Your main request, i.e., the content handler
> should not send headers
> > or any contents by itself and should return
> NGX_DONE.
> >
>
> I was a bit hand-waving here. To be more accurate:
>
> In the "post_subrequest" handler, sends headers
> and contents directly
> on "r->parent" rather than "r" because the latter
> references the
> current subrequest while the former is our "main
> request". Also never
> forget to call ngx_http_finalize_request on
> "r->parent", or you'll
> observe that "mysterious hang" for nginx 0.8.21+
> mentioned in one of
> my earlier replies ;)
>
> > I think this model should work after my various
> experiments in the
> > "echo" module, but I could be wrong ;) Feel free
> to tell us your
> > findings on your way :)
> >
>
> Now I believe it should work ;)
>
> Thanks for making me thinking about this issue
> very hard again :)
>
> Best,
> -agentzh

Well, I've got everything *almost* working. Not calling ngx_http_finalize_request on the parent request cost me a couple of hours of debugging mysterious hangs when there were multiple subrequests, but after doing that the hang goes away and the output looks good. So far, it works perfectly if all the requests return immediately, if all the requests error out and if all the requests but one error out. The only remaining issue I'm having is when there are 2 or more subrequests the main request does return until both subrequests finish. In my test case, I have one subrequest return after 0.1 seconds and the other after 10 seconds. It returns the output from the fast request, as expected, but it returns it after 10 seconds. Very strange.

I'm guessing I need to do something special to cancel the pending subrequests, or possibly force the completion of the parent request regardless of the state of the rest of subrequests. I tried calling ngx_http_finalize_request with various rc values on the pending subrequests, hoping to find something that would force the completion, but nothing seemed to work (I can get the parent request to close immediately, but that doesn't really help since it hasn't sent the good request's response yet).

Is there a way to cancel pending subrequests or to keep the parent request from being postponed? Either option would probably work.

Thanks for the help so far -- I've made a lot of progress. If I can sort out this last bit, I should have a fully working prototype tomorrow. (and on our production machines by Friday! ;)

--Shaun
Subject Author Posted

Parallel subrequests for multi-source long polling?

shaun November 18, 2009 01:30AM

Re: Parallel subrequests for multi-source long polling?

Piotr Sikora November 18, 2009 02:28AM

Re: Parallel subrequests for multi-source long polling?

agentzh November 18, 2009 03:28AM

Re: Parallel subrequests for multi-source long polling?

agentzh November 18, 2009 03:30AM

Re: Parallel subrequests for multi-source long polling?

shaun November 18, 2009 04:22AM

Re: Parallel subrequests for multi-source long polling?

agentzh November 18, 2009 11:30PM

Re: Parallel subrequests for multi-source long polling?

shaun November 19, 2009 03:54AM

Re: Parallel subrequests for multi-source long polling?

agentzh November 19, 2009 04:58AM

Re: Parallel subrequests for multi-source long polling?

agentzh November 19, 2009 06:32AM

Re: Parallel subrequests for multi-source long polling?

shaun November 19, 2009 02:20PM

Re: Parallel subrequests for multi-source long polling?

Piotr Sikora November 19, 2009 03:28PM

Re: Parallel subrequests for multi-source long polling?

shaun November 19, 2009 04:08PM

Re: Parallel subrequests for multi-source long polling?

Piotr Sikora November 19, 2009 05:42PM

Re: Parallel subrequests for multi-source long polling?

shaun November 20, 2009 03:33AM

Re: Parallel subrequests for multi-source long polling?

Piotr Sikora November 20, 2009 04:48AM

Re: Parallel subrequests for multi-source long polling?

agentzh November 20, 2009 05:28AM

Re: Parallel subrequests for multi-source long polling?

shaun November 20, 2009 06:41AM

Re: Parallel subrequests for multi-source long polling?

shaun November 21, 2009 09:12PM

Re: Parallel subrequests for multi-source long polling?

Piotr Sikora November 21, 2009 10:10PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 84
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready