Hello!
On Mon, May 29, 2017 at 06:04:05AM -0400, isolomka wrote:
> Hi All,
> I'm developing NGINX module which should read some binary data from upstream
> server, process it an send response to the user.
> So to do it i'm using ngx_http_upstream API to create upstream request and
> get data. So the question is where is the right point to start processing of
> received chain of buffers and how to avoid call to ngx_http_output_filter
> from ngx_http_upstream_process_non_buffered_request?
> I'm going to cal it by myself when data processing is finished.
When using non-buffered mode, there are two basic callbacks you
should care about (u == u->upstream):
- u->process_header, which is to parse response headers; you are
expected to return NGX_OK when all headers are parsed, and the
remaining data is request body.
- u->input_filter, which is to convert input data from upstream
(as available in u->buffer) to output buffers to be sent to
client. You are expected to store output buffers in the
u->out_bufs chain. If there is not enough data to process you
can simply return NGX_OK without moving the u->buffer pointers,
but note that buffer size is finite.
Trying to call ngx_http_output_filter() is not going to work, you
are expected to follow the interface available.
You may want to read the memcached module sources for additional
details (src/http/modules/ngx_http_memcached_module.c), it is
relatively simple to understand.
> During research i've tried to enable subrequest_in_memory flag to save all
> needed data for later processing
> In upstream ngx_http_upstream_process_header there is a validation if
> subrequest is in memory
> if (!r->subrequest_in_memory) {
> ngx_http_upstream_send_response(r, u);
> return;
> }
> When i enabled subrequest_in_memory flag i got an error "upstream buffer is
> too small to read response"
> But in debugger i see that data are available in the upstream buffer. It
> looks for me that response is already saved. So how to explain such
> inconsistency?
The subrequest_in_memory flag is expected to be used when a
subrequest is asked to preserve response content in u->buffer
instead of sending it to the client. This is used primary by the
"include" SSI command when used with the "set" parameter, and
requires a response to fit into the configured buffer. It is
expected to just work with simple u->input_filter implementations,
and may require special additional handling in case of complex
protocols to provide continuous data in u->buffer (see
ngx_http_proxy_non_buffered_chunked_filter() for an example).
Trying to set r->subrequest_in_memory from a protocol
implementation is wrong, and may result in problems. The
particular message you are seeing suggests that the response you
are trying to process is bigger than the buffer configured.
As suggested above, consider looking into the memcached module
sources for a simple example of a module using upstream
infrastructure.
--
Maxim Dounin
http://nginx.org/
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
