Welcome! Log In Create A New Profile

Advanced

Thread Pool HTTP POST issue

December 16, 2020 09:36AM
Hi,we are doing some tests using the nginx thread pool feature. The scenario is the following:1) the worker process register with ngx_http_read_client_request_body fuction a handler for the request. We can call it "post_handler"
2) the worker process receive a POST bid request. The payload is a json that contains an id field (unique among requests that will be used in the response generated by our web service)
3) the worker process in the "post_handler" create the task context and launch the task
4) the request payload is read in the task code that is executed in the thread pool
5) the blocking code is executed in the task code that is executed in the thread pool
6) task completion handler (worker process - main thread - outside thread pool):
the ngx_http_send_header(r)
ngx_http_output_filter(r, out); ngx_http_finalize_request(r, r->headers_out.status); code is executed in the task completion handler Now some details:
* in point 3:
-- the "r" (ngx_http_request_t) pointer and "b" (ngx_buf_t *) the response buffer pointer are saved as fields in the task context (a simple c structure used as the context of the task)
-- just before the code for the task launch (call to "ngx_thread_task_post" function) we put the following lines of code:
r->main->blocked++;
r->aio = 1;* in point 5, after the blocking code execution and after the generated response string has been saved in a task context field,
-- the buffer output chain is allocated into the heap and stored into a task context field:
//buffer chain needed to put response buffer in
ngx_chain_t *out = ngx_alloc_chain_link(r->pool); -- the buffers are filled in (r, b were stored into the task context before task launch - response is a local var where the response saved into the task context has been copied):
r->headers_out.content_type_len = sizeof("application/json") - 1;
r->headers_out.content_type.data = (u_char *) "application/json"; r->headers_out.content_length_n = strlen(response); /* adjust the pointers of the buffer */
b->pos = (u_char *)response;
b->last = (u_char *)response + strlen(response); b->memory = 1; /* this buffer is in memory */
b->last_buf = 1; /* this is the last buffer in the buffer chain */
(*out).buf = b;
(*out).next = NULL;* in point 6, just before the instructions described above (for point 6) we put the following lines of code:
r->main->blocked--;
r->aio = 0; Issue we are trying to fix:- in point 6 comparing the response string saved into the task context with the buffer chain "out" var content (the comparison is done just before calling "ngx_http_output_filter" and "ngx_http_finalize_request") we discovered that about the 40% of the times the 2 values are different for two reasons:
- "out" content is an empty string
- "out" content is a response with id different than the one of the response string in the task context (that is the one used to fill in "out" buffer chain in point 5)We think there is probably an error either about what it is possible to put into the task context structure (maybe it is not possible to manipulate "r", "b", "out" outside the main thread...) or the right usage and position of "r->main->blocked" counter and "r->aio" flag.
Any hints about how to solve this issue?Thanks._______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

Thread Pool HTTP POST issue

st.gabrielli December 16, 2020 09:36AM

Re: Thread Pool HTTP POST issue

Maxim Dounin December 16, 2020 12:22PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 63
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready