Welcome! Log In Create A New Profile

Advanced

Re: Parallel subrequests for multi-source long polling?

November 19, 2009 02:20PM
agentzh Wrote:
-------------------------------------------------------
> On Thu, Nov 19, 2009 at 5:54 PM, agentzh wrote:
> >
> > It's weird that the client sees the response
> header and body after the
> > slowest subrequest finishes. It seems that the
> response has been
> > buffered in the last few output filters somehow.
> Please ensure that
> > you have set b->flush and b->last_buf in your
> output chain link. These
> > flags should defeat buffering in most cases.
> >
>
> Okay, I was wrong here :) The output body of the
> current subrequest
> and its parent request will be permanently
> buffered if the request
> under question is not at the current head of the
> postponed chain. For
> example
>
> location /main {
> echo hello;
> echo_flush;
> echo_location_async '/foo';
> echo_location_async '/bar';
> echo_location_async '/baz';
> echo world;
> echo_flush;
> }
>
> location /foo {
> echo_sleep 1;
> echo foo;
> echo_flush;
> }
>
> location /bar {
> echo_sleep 2;
> echo bar;
> echo_flush;
> }
>
> location /baz {
> echo_sleep 1;
> echo baz;
> echo_flush;
> }
>
> Accessing /main using curl will show "hello"
> immediately, and "foo" 1
> sec later, and finally "bar", "baz", and "world"
> together after
> another 1 sec. So if the slowest subrequest is
> issued first, like the
> location /foo here, then there's little hope to
> get the outputs of
> later subrequests like /baz properly flushed
> without using hacks.
>
> The "world" output of the main request is buffered
> because it's at the
> end of the postponed chain while "hello" is at the
> head. When "hello"
> gets out, "foo" becomes the head of the postponed
> chain.
>
> > And still, we have to cancel the pending
> subrequests, but close the
> > connection seems a bit overkill especially in
> the context of HTTP
> > keepalive.
> >
>
> Forcibly cancle subrequests can be dangerous
> because we have to ensure
> all those timers and event handlers get properly
> cleared. And I have
> never done such things myself. Sorry about that.
> chaoslawful and I
> will take a closer look at this issue but with no
> promise.
>
> >
> >> I tried calling ngx_http_finalize_request with
> various rc values on the pending subrequests,
> hoping to find something that would force the
> completion, but nothing seemed to work
> >
>
> Yeah, it won't work. I've tested within my "echo"
> module by
> introducing a "echo_abort_parent" directive. I
> wonder if we'll have to
> arrange the postponed chain ourselves or bypass
> the postpone filter
> completely. I'm not sure. It's getting evil
> already :)
>
> Maybe other people on the list can give some
> advice on canceling a
> pending subrequest?
>
> Cheers,
> -agentzh

My plan is to start messing with the postpone chain and see if I can rearrange it in a way that won't totally break.

So, as another option, I wonder if it's possible to do this using completely separate requests. I have no idea about the feasibility of this, but it seems like it could work. I could create new requests where I was using subrequests before and then configure the upstream manually, attach a context and then collate the results in the output filter. Everything would work nearly the same except the finalizing of the main request would have to happen in the child request filter instead of the subrequest's postreq handler.

I'll take a stab at hacking the postpone chains and if that doesn't seem safe or sane, I'll give the second option a try.

Oh, I'll also try to get the code up on github a bit later today as well.

--Shaun
Subject Author Posted

Parallel subrequests for multi-source long polling?

shaun November 18, 2009 01:30AM

Re: Parallel subrequests for multi-source long polling?

Piotr Sikora November 18, 2009 02:28AM

Re: Parallel subrequests for multi-source long polling?

agentzh November 18, 2009 03:28AM

Re: Parallel subrequests for multi-source long polling?

agentzh November 18, 2009 03:30AM

Re: Parallel subrequests for multi-source long polling?

shaun November 18, 2009 04:22AM

Re: Parallel subrequests for multi-source long polling?

agentzh November 18, 2009 11:30PM

Re: Parallel subrequests for multi-source long polling?

shaun November 19, 2009 03:54AM

Re: Parallel subrequests for multi-source long polling?

agentzh November 19, 2009 04:58AM

Re: Parallel subrequests for multi-source long polling?

agentzh November 19, 2009 06:32AM

Re: Parallel subrequests for multi-source long polling?

shaun November 19, 2009 02:20PM

Re: Parallel subrequests for multi-source long polling?

Piotr Sikora November 19, 2009 03:28PM

Re: Parallel subrequests for multi-source long polling?

shaun November 19, 2009 04:08PM

Re: Parallel subrequests for multi-source long polling?

Piotr Sikora November 19, 2009 05:42PM

Re: Parallel subrequests for multi-source long polling?

shaun November 20, 2009 03:33AM

Re: Parallel subrequests for multi-source long polling?

Piotr Sikora November 20, 2009 04:48AM

Re: Parallel subrequests for multi-source long polling?

agentzh November 20, 2009 05:28AM

Re: Parallel subrequests for multi-source long polling?

shaun November 20, 2009 06:41AM

Re: Parallel subrequests for multi-source long polling?

shaun November 21, 2009 09:12PM

Re: Parallel subrequests for multi-source long polling?

Piotr Sikora November 21, 2009 10:10PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 115
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready