Welcome! Log In Create A New Profile

Advanced

Processing responses of unbounded sizes from upstream servers

Maxime Henrion
October 22, 2015 08:32AM
Hello all,


I am currently developing a module that has to send a number of subrequests to upstream servers, and aggregate them through application logic. I am currently doing that through a post-subrequest handler, using the NGX_HTTP_SUBREQUEST_IN_MEMORY flag. My problem is that it is possible to receive very large responses from the upstream servers, and I end up with the "upstream buffer is too small" error, even after bumping the buffer sizes a number of times.

It is my understanding that if I drop this subrequest flag, nginx wouldn't try to make the response fit in a single buffer anymore, but then I have no idea how to get at that buffer chain - my post-subrequest handler only knows about the single buffer in the upstream structure and I haven't been able to locate a piece of code that would do things differently.

I suppose it would be possible to use an output filter instead of a post-subrequest handler for that use case, would that make sense? And last but not least, if I go down that road, can I just move my module "declaration" to the HTTP_AUX_FILTER_MODULES variable (from HTTP_MODULES), and still have the rest of module work fine, or will I need to use a second module for that?

Thanks a lot in advance!
Maxime
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel
Subject Author Views Posted

Processing responses of unbounded sizes from upstream servers

Maxime Henrion 537 October 22, 2015 08:32AM

Re: Processing responses of unbounded sizes from upstream servers

Sorin Manole 264 October 26, 2015 03:08PM

RE: Processing responses of unbounded sizes from upstream servers

Maxime Henrion 245 October 27, 2015 12:02PM

RE: Processing responses of unbounded sizes from upstream servers

Maxime Henrion 262 October 27, 2015 12:14PM

Re: Processing responses of unbounded sizes from upstream servers

Maxim Dounin 242 October 27, 2015 12:26PM

RE: Processing responses of unbounded sizes from upstream servers

Maxime Henrion 244 October 27, 2015 01:06PM

Re: Processing responses of unbounded sizes from upstream servers

Maxim Konovalov 270 November 02, 2015 05:08AM

Re: Processing responses of unbounded sizes from upstream servers

Maxim Konovalov 262 November 02, 2015 05:50AM

Re: Processing responses of unbounded sizes from upstream servers

Hung Nguyen 278 November 02, 2015 05:56AM



Sorry, you do not have permission to post/reply in this forum.

Online Users

Guests: 232
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready