Welcome! Log In Create A New Profile

Advanced

Re: Upstream module usage to process data

Maxim Dounin
May 30, 2017 03:00PM
Hello!

On Tue, May 30, 2017 at 03:45:15AM -0400, isolomka wrote:

> Hi Maxim,
> Thanks for quick response.
> I've implemented all upstream callbacks and upstream seems to work fine
> now.
>
> But i still have the open question how to avoid sending received data from
> upstream to the downstream client.
> As I said, I need to process received data first and after that send result
> to output.
>
> As i see in the ngx_http_upstream_process_non_buffered_request() buffers
> are sent to output if out_bufs are not empty:
> if (u->out_bufs || u->busy_bufs) {
> rc = ngx_http_output_filter(r, u->out_bufs);
>
> So is it a normal design from nginx point of view to store data in own
> buffer (not in u->out_bufs ) to avoid call to ngx_http_output_filter?
> I will call it later with my own chain of buffers when all data will be
> correctly processed.

There are two basic options you can use:

- Keep the data in u->buffer untill you can do appropriate process,
then put the result into u->out_bufs. This will work as long as
u->buffer is big enough for responses you are expected to
handle. For example, this what memcached module does when it
parses a response trailer.

- Copy the data elsewhere. Once the response is complete, process
it, then put to u->out_bufs.

Trying to call ngx_http_output_filter() yourself is not likely to
work correctly, and certainly not how input filters are expected
to work.

Note well that if you need to do sophisticated processing of input
data in an input filter, it might indicate that you are using it
wrong. Input filters are to remove various protocol-dependent
transport encapsulation of data, such as chunked transfer encoding
in HTTP and/or FastCGI records. If you want to change data
representation from one form to another, implementing this as a
response body filter might be better solution, see
http://nginx.org/en/docs/dev/development_guide.html#http_body_filters.

--
Maxim Dounin
http://nginx.org/
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

Upstream module usage to process data

isolomka May 29, 2017 06:04AM

Re: Upstream module usage to process data

Maxim Dounin May 29, 2017 07:58AM

Re: Upstream module usage to process data

isolomka May 30, 2017 03:45AM

Re: Upstream module usage to process data

Maxim Dounin May 30, 2017 03:00PM

Re: Upstream module usage to process data

isolomka May 31, 2017 08:22AM

Re: Upstream module usage to process data

Maxim Dounin May 31, 2017 08:54AM

Re: Upstream module usage to process data

isolomka May 31, 2017 09:00AM

Re: Upstream module usage to process data

Maxim Dounin May 31, 2017 09:22AM

Re: Upstream module usage to process data

isolomka May 31, 2017 09:25AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 177
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready