Welcome! Log In Create A New Profile

Advanced

Asynchronous file processing

June 05, 2014 12:23PM
Hi all,

I'm working on a native nginx module in which I want to read an input file and perform some manipulations on its data. Since the files I'm reading are big and accessed over NFS, I want to use asynchronous I/O for reading them, and I want to implement it as a pipeline of chunks, i.e. read a chunk, process the chunk, add the chunk to the response chain.

My questions are as follows:
1. Is there any sample / documentation / basic guidelines you can give me on how to progressively output data back to the client ?
2. Does nginx support generating responses with chunked-encoding (I may not be able to determine the response size without processing the whole file, and would prefer to avoid keeping the entire processed buffer in memory at once) ?
3. Can you send me some guidelines on how to perform asynchronous file read (my understanding is that I need to use ngx_file_aio_read and if I get NGX_AGAIN, set the handler/data members of file->aio to my completion callback) ?

Thank you in advance,

Eran
Subject Author Posted

Asynchronous file processing

erankor2 June 05, 2014 12:23PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 276
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready