Welcome! Log In Create A New Profile

Advanced

Asynchronous file processing

June 05, 2014 12:23PM
Hi all,

I'm working on a native nginx module in which I want to read an input file and perform some manipulations on its data. Since the files I'm reading are big and accessed over NFS, I want to use asynchronous I/O for reading them, and I want to implement it as a pipeline of chunks, i.e. read a chunk, process the chunk, add the chunk to the response chain.

My questions are as follows:
1. Is there any sample / documentation / basic guidelines you can give me on how to progressively output data back to the client ?
2. Does nginx support generating responses with chunked-encoding (I may not be able to determine the response size without processing the whole file, and would prefer to avoid keeping the entire processed buffer in memory at once) ?
3. Can you send me some guidelines on how to perform asynchronous file read (my understanding is that I need to use ngx_file_aio_read and if I get NGX_AGAIN, set the handler/data members of file->aio to my completion callback) ?

Thank you in advance,

Eran
SubjectAuthorPosted

Asynchronous file processing

erankor2June 05, 2014 12:23PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 95
Record Number of Users: 7 on March 06, 2014
Record Number of Guests: 184 on July 08, 2014
Powered by nginx    Powered by FreeBSD    PHP Powered    Powered by Percona     ipv6 ready