here is a synthetic test on vm, not perfect, but representative:


[root@nginx-single ~]# dd if=/dev/zero of=/writetest bs=8k count=30960
30960+0 records in
30960+0 records out
253624320 bytes (254 MB) copied, 0.834861 s, 304 MB/s

[root@nginx-single ~]# dd if=/dev/zero of=/writetest bs=8k count=30960 conv=fsync
30960+0 records in
30960+0 records out
253624320 bytes (254 MB) copied, 0.854208 s, 297 MB/s

[root@nginx-single ~]# dd if=/dev/zero of=/writetest bs=8k count=61960
61960+0 records in
61960+0 records out
507576320 bytes (508 MB) copied, 1.71833 s, 295 MB/s
[root@nginx-single ~]# dd if=/dev/zero of=/writetest bs=8k count=61960 conv=fsync
61960+0 records in
61960+0 records out
507576320 bytes (508 MB) copied, 1.74482 s, 291 MB/s


br,
Aziz.





> On 1 Mar 2018, at 00:41, Aziz Rozyev <arozyev@nginx.com> wrote:
>
> Valery,
>
> may you please suggest how you came to the conclusion that
>
> “fsync simply instructs OS to ensure consistency of a file"?
>
> As far as understand simply instructing OS staff come at no cost, right?
>
>> Without fsyncing file's data and metadata a client will receive a positive reply before data has reached the storage, thus leaving non-zero probability that states of two systems involved into a web transaction end up inconsistent.
>
>
> I understand why one may need consistency, but doing so with fsyncing is non-sense.
>
> Here is what man page says in that regard:
>
>
> fsync() transfers ("flushes") all modified in-core data of (i.e., modified buffer cache pages for) the file referred to by the file descriptor fd to the disk device (or other permanent
> storage device) so that all changed information can be retrieved even after the system crashed or was rebooted. This includes writing through or flushing a disk cache if present. The call
> blocks until the device reports that the transfer has completed. It also flushes metadata information associated with the file (see stat(2)).
>
>
>
>
> br,
> Aziz.
>
>
>
>
>
>> On 28 Feb 2018, at 21:24, Valery Kholodkov <valery+nginxen@grid.net.ru> wrote:
>>
>> It's completely clear why someone would need to flush file's data and metadata upon a WebDAV PUT operation. That is because many architectures expect a PUT operation to be completely settled before a reply is returned.
>>
>> Without fsyncing file's data and metadata a client will receive a positive reply before data has reached the storage, thus leaving non-zero probability that states of two systems involved into a web transaction end up inconsistent.
>>
>> Further, the exact moment when the data of certain specific file reaches the storage depends on numerous factors, for example, I/O contention. Consequently, the exact moment when the data of a file being uploaded reaches the storage can be only determined by executing fsync.
>>
>> val
>>
>> On 28-02-18 11:04, Aziz Rozyev wrote:
>>> While it’s not clear why one may need to flush the data on each http operation,
>>> I can imagine to what performance degradation that may lead of.
>>> if it’s not a some kind of funny clustering among nodes, I wouldn't care much
>>> where actual data is, RAM still should be much faster, than disk I/O.
>>> br,
>>> Aziz.
>>>> On 28 Feb 2018, at 12:30, Nagy, Attila <bra@fsn.hu> wrote:
>>>>
>>>> On 02/27/2018 02:24 PM, Maxim Dounin wrote:
>>>>>
>>>>>> Now, that nginx supports running threads, are there plans to convert at
>>>>>> least DAV PUTs into it's own thread(pool), so make it possible to do
>>>>>> non-blocking (from nginx's event loop PoV) fsync on the uploaded file?
>>>>> No, there are no such plans.
>>>>>
>>>>> (Also, trying to do fsync() might not be the best idea even in
>>>>> threads. A reliable server might be a better option.)
>>>>>
>>>> What do you mean by a reliable server?
>>>> I want to make sure when the HTTP operation returns, the file is on the disk, not just in a buffer waiting for an indefinite amount of time to be flushed.
>>>> This is what fsync is for.
>>>>
>>>> Why doing this in a thread is not a good idea? It would'nt block nginx that way.
>>
>> _______________________________________________
>> nginx mailing list
>> nginx@nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
>

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx