Hi Sergey,

thank you for your answer.

When I'm hardcoding the value of `fastcgi_param CONTENT_LENGTH` to the
expected length for my test request, the request's body *is* available.
That does confirm your thoughts on the backend expecting a length
indication.

However, I'm not sure if your if your assumption about FastCGI being
inherently chunked using their records is correct.

The spec says [0]

> Next the Responder application receives CGI/1.1 stdin data from the
Web server over FCGI_STDIN. The application receives at most
CONTENT_LENGTH bytes from this stream before receiving the end-of-stream
indication. (The application receives less than CONTENT_LENGTH bytes
only if the HTTP client fails to provide them, e.g. because the client
crashed.)

Considering FastCGI is an extension to CGI, is it supposed to conform to
the CGI spec as well? There it says

> The server MUST set this meta-variable if and only if the request is
> accompanied by a message-body entity. The CONTENT_LENGTH value must
> reflect the length of the message-body after the server has removed
> any transfer-codings or content-codings.

Our FastCGI backend, php [2], does rely on the CONTENT_LENGTH parameter.
This is in line with your remark, that our backend expects it.

While I see that it is possible to implement reading the request's body
without the CONTENT_LENGHT value, I am not sure how to tackly this
problem exactly. At this point, I'd say this is either an nginx bug, due
to the fact that it streams the body and does not set a CONTENT_LENGTH
value, or a bug in php-fpm, due to the fact that it relies on this
value. What do you think?


[0] http://www.mit.edu/~yandros/doc/specs/fcgi-spec.html#S6.2
[1] https://datatracker.ietf.org/doc/html/rfc3875#section-4.1.2
[2]
https://github.com/php/php-src/blob/1f9b044c809159b90a2298aa462346131d1c1207/sapi/fpm/fpm/fpm_main.c#L432


On 13.09.21 13:51, Sergey Kandaurov wrote:
>
>> On 13 Sep 2021, at 12:32, Tim Siebels <tim.siebels@iserv.eu> wrote:
>>
>> Hi everyone,
>>
>> We are experiencing an issue when using `fastcgi_request_buffering off;` in combination with chunked transfer-encoding. The application will not receive any body. Depending on the system, we *sometimes* receive a body. Enabling `fastcgi_request_buffering` fixes the issue. However, we would like not to enable this for every request.
>>
>> We are using Debian Bullseye [0]. Additionally, I was able to reproduce this issue using a self compiled 1.21.3 [1].
>>
>> We are using a very simple configuration [2], only disabling `fastcgi_request_buffering`. The application is using PHP over fastcgi [3].
>>
>> Furthermore, we log `$request_body` in a custom log file.
>> Whenever this logfile contains the expected request body, the application receives the expected body. As far as I understand, this cannot work without buffering. I assume this to be an in-memory buffer. This goes in line with the body not being passed, when we increase the size. The implementation of `ngx_http_read_client_request_body` does have an optimization, if the entire body fits into header_in.
>> `sleep`ing between chunks also removes the possibility that the body is passed to the application.
>> We could not reproduce that the body is *sometimes* passed in a self-contained docker container or using a self-compiled version of nginx.
>> These never work with fastcgi_request_buffering off. At least not, if the body is large enough.
>
>
>>
>> Apache had a bug that had a similar effect [4,5,6].
>> Our understanding from these bugreports is, that the fastcgi protocol expects a defined content-length to be able to read the request body.
>
> FastCGI is essentially a protocol with builtin chunked encoding,
> see http://www.mit.edu/~yandros/doc/specs/fcgi-spec.html#S3.3.
> Hence, the Content-Length header is rather complementary (like in HTTP/2),
> e.g. it allows to communicate the message length knowledge in advance.
>
> So, when nginx doesn't buffer request body, and it's not wholly present
> in the request body memory buffers at the time of forming request headers,
> in this case the length cannot be calculated for chunked body requests.
> See also the description of $content_length: nginx.org/r/$content_length
>
>> Our conclusion is, that nginx is supposed to buffer all request with a chunked transfer-encoding, when using fastcgi. Regardless of the fastcgi_request_buffering option. A comment on a older, unrelated, bugreport for nginx confirms this [7].
>> Therefore, we expect this to be a supported use case.
>
> When a client uses chunked transfer encoding, and nginx is configured
> to pass requests to fastcgi with disabled fastcgi_request_buffering,
> the intent is to pass request body chunks to backend (with conversion
> to fastcgi records) as soon as they are received from a client.
>
>>
>> There are some bug reports out there experiencing similar issues [8,9].
>>
>
> Looks like they are suffering from the same sort of a problem.
>
>> We tried to set `client_body_in_file_only` to `on` to be able to see the buffer files. However, these are never created with buffering off.
>>
>
> It is disabled in the unbuffered mode.
>
>> error.log is empty.
>>
>> Can anyone help us how to debug this further?
>
> For a start, you may want to examine what is actually passed to backend.
>
>>
>> Thanks,
>> Tim
>>
>> [..]
>> [2]
>> user nginx;
>> worker_processes auto;
>> pid /run/nginx.pid;
>>
>> events {
>> worker_connections 768;
>> }
>>
>> http {
>> access_log /var/log/nginx/access.log;
>> error_log /var/log/nginx/error.log;
>>
>> fastcgi_request_buffering off;
>>
>> include /etc/nginx/sites-enabled/*;
>> }
>>
>> [3]
>> log_format postdata '"$request" $status $request_body ($request_length)';
>>
>> server {
>> listen *:982;
>>
>> location /iserv/helloworld {
>> root /usr/share/iserv/helloworld/public;
>>
>> access_log /var/log/nginx/postdata.log postdata;
>>
>> fastcgi_param QUERY_STRING $query_string;
>> fastcgi_param REQUEST_METHOD $request_method;
>> fastcgi_param CONTENT_TYPE $content_type;
>> fastcgi_param CONTENT_LENGTH $content_length;
>
> Looks like the backend is expecting to receive something
> in the CONTENT_LENGTH header to read the request body.
>
> The best solution is to teach your FastCGI backend how to receive
> the request body without CONTENT_LENGTH.
>
>>
>> fastcgi_param SCRIPT_NAME $fastcgi_script_name;
>> fastcgi_param REQUEST_URI $request_uri;
>> fastcgi_param DOCUMENT_URI $document_uri;
>> fastcgi_param DOCUMENT_ROOT $document_root;
>>
>> fastcgi_param SCRIPT_FILENAME $document_root/index.php$fastcgi_script_name;
>>
>> fastcgi_pass unix:/run/php/php-fpm.iserv-helloworld.sock;
>> }
>>
>> }
>>
>> [4] https://bz.apache.org/bugzilla/show_bug.cgi?id=53332
>> [5] https://bz.apache.org/bugzilla/show_bug.cgi?id=57087
>> [6] https://bugs.php.net/bug.php?id=60826
>> [7] https://trac.nginx.org/nginx/ticket/1344
>> [8] https://trac.cyberduck.io/wiki/help/en/howto/mount/issues/fastcgi
>> [9] https://github.com/nextcloud/server/issues/7995
>> _______________________________________________
>> nginx mailing list
>> nginx@nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
>
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx