Welcome! Log In Create A New Profile

Advanced

Re: fsync()-in webdav PUT

Aziz Rozyev
February 28, 2018 04:44PM
Valery,

may you please suggest how you came to the conclusion that

“fsync simply instructs OS to ensure consistency of a file"?

As far as understand simply instructing OS staff come at no cost, right?

> Without fsyncing file's data and metadata a client will receive a positive reply before data has reached the storage, thus leaving non-zero probability that states of two systems involved into a web transaction end up inconsistent.


I understand why one may need consistency, but doing so with fsyncing is non-sense.

Here is what man page says in that regard:


fsync() transfers ("flushes") all modified in-core data of (i.e., modified buffer cache pages for) the file referred to by the file descriptor fd to the disk device (or other permanent
storage device) so that all changed information can be retrieved even after the system crashed or was rebooted. This includes writing through or flushing a disk cache if present. The call
blocks until the device reports that the transfer has completed. It also flushes metadata information associated with the file (see stat(2)).




br,
Aziz.





> On 28 Feb 2018, at 21:24, Valery Kholodkov <valery+nginxen@grid.net.ru> wrote:
>
> It's completely clear why someone would need to flush file's data and metadata upon a WebDAV PUT operation. That is because many architectures expect a PUT operation to be completely settled before a reply is returned.
>
> Without fsyncing file's data and metadata a client will receive a positive reply before data has reached the storage, thus leaving non-zero probability that states of two systems involved into a web transaction end up inconsistent.
>
> Further, the exact moment when the data of certain specific file reaches the storage depends on numerous factors, for example, I/O contention. Consequently, the exact moment when the data of a file being uploaded reaches the storage can be only determined by executing fsync.
>
> val
>
> On 28-02-18 11:04, Aziz Rozyev wrote:
>> While it’s not clear why one may need to flush the data on each http operation,
>> I can imagine to what performance degradation that may lead of.
>> if it’s not a some kind of funny clustering among nodes, I wouldn't care much
>> where actual data is, RAM still should be much faster, than disk I/O.
>> br,
>> Aziz.
>>> On 28 Feb 2018, at 12:30, Nagy, Attila <bra@fsn.hu> wrote:
>>>
>>> On 02/27/2018 02:24 PM, Maxim Dounin wrote:
>>>>
>>>>> Now, that nginx supports running threads, are there plans to convert at
>>>>> least DAV PUTs into it's own thread(pool), so make it possible to do
>>>>> non-blocking (from nginx's event loop PoV) fsync on the uploaded file?
>>>> No, there are no such plans.
>>>>
>>>> (Also, trying to do fsync() might not be the best idea even in
>>>> threads. A reliable server might be a better option.)
>>>>
>>> What do you mean by a reliable server?
>>> I want to make sure when the HTTP operation returns, the file is on the disk, not just in a buffer waiting for an indefinite amount of time to be flushed.
>>> This is what fsync is for.
>>>
>>> Why doing this in a thread is not a good idea? It would'nt block nginx that way.
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

fsync()-in webdav PUT

Nagy, Attila February 27, 2018 05:24AM

Re: fsync()-in webdav PUT

Maxim Dounin February 27, 2018 08:26AM

Re: fsync()-in webdav PUT

Nagy, Attila February 28, 2018 04:32AM

Re: fsync()-in webdav PUT

Aziz Rozyev February 28, 2018 05:06AM

Re: fsync()-in webdav PUT

Valery Kholodkov February 28, 2018 01:26PM

Re: fsync()-in webdav PUT

Aziz Rozyev February 28, 2018 04:44PM

Re: fsync()-in webdav PUT

Aziz Rozyev February 28, 2018 05:08PM

Re: fsync()-in webdav PUT

pbooth February 28, 2018 05:36PM

Re: fsync()-in webdav PUT

Valery Kholodkov March 01, 2018 07:26AM

Re: fsync()-in webdav PUT

Valery Kholodkov March 01, 2018 07:30AM

Re: fsync()-in webdav PUT

Nagy, Attila March 02, 2018 04:02AM

Re: fsync()-in webdav PUT

Nagy, Attila March 02, 2018 05:04AM

Re: fsync()-in webdav PUT

Nagy, Attila March 02, 2018 03:52AM

Re: fsync()-in webdav PUT

Maxim Dounin February 28, 2018 09:10AM

Re: fsync()-in webdav PUT

Valery Kholodkov February 28, 2018 01:56PM

Re: fsync()-in webdav PUT

itpp2012 February 28, 2018 03:28PM

Re: fsync()-in webdav PUT

Nagy, Attila March 02, 2018 04:14AM

Re: fsync()-in webdav PUT

Aziz Rozyev March 02, 2018 05:44AM

Re: fsync()-in webdav PUT

Nagy, Attila March 02, 2018 06:32AM

RE: fsync()-in webdav PUT

Reinis Rozitis March 04, 2018 07:42AM

Re: fsync()-in webdav PUT

Nagy, Attila March 05, 2018 05:32AM

RE: fsync()-in webdav PUT

Reinis Rozitis March 05, 2018 06:56AM

Re: fsync()-in webdav PUT

Valery Kholodkov March 05, 2018 08:14AM

Re: fsync()-in webdav PUT

Nagy, Attila March 05, 2018 08:56AM

Re: fsync()-in webdav PUT

Richard Demeny March 05, 2018 09:06AM

Re: fsync()-in webdav PUT

Maxim Dounin March 02, 2018 11:08AM

Re: fsync()-in webdav PUT

Valery Kholodkov March 02, 2018 02:50PM

Re: fsync()-in webdav PUT

Maxim Dounin March 02, 2018 07:44PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 186
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 500 on July 15, 2024
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready