Welcome! Log In Create A New Profile

Advanced

RE: cache: move open to thread pool

August 08, 2018 03:46PM
Hi,

>
> - The code bypasses open file cache, and uses a direct call
> in the http cache code instead. While it might be ok in your
> setup, it looks like an incomplete solution from the generic point
> of view. A better solution would be to introduce a generic
> interface in ngx_open_cached_file() to allow use of thread
> pools.
>

A small comment on this - I wrote such an implementation a while ago
in my module. We've been using it on production for the last ~3 years.

Code is here -
https://github.com/kaltura/nginx-vod-module/blob/master/ngx_async_open_file_cache.c

(There's a lot of code there that was copied from ngx_open_file_cache.c,
I did that since those functions are static, and didn't want to change nginx core.
If implemented as part of nginx core, all this duplication can be avoided...)

In my implementation, I added a function similar to ngx_open_cached_file -
(ngx_async_open_cached_file), that gets a few extra params -
1. The thread pool
2. The thread task (optional) - this was done in order to reuse the task
when opening multiple files in a single request
3. Callback + context - invoked once the async open completes

This function first checks the cache, if there's a hit, it returns synchronously
(NGX_OK).
Otherwise, it posts a task that does ngx_open_and_stat_file (NGX_AGAIN).
When the task completes, the main nginx thread updates the cache, and invokes
the user callback.

Hope this helps...

Eran

_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel
Subject Author Views Posted

cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 1152 August 07, 2018 05:28PM

Re: cache: move open to thread pool

Maxim Dounin 248 August 08, 2018 02:18PM

RE: cache: move open to thread pool

erankor 241 August 08, 2018 03:46PM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 187 August 27, 2018 06:00PM

RE: cache: move open to thread pool

erankor 210 August 28, 2018 05:18AM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 206 August 09, 2018 05:44PM

Re: cache: move open to thread pool

Maxim Dounin 552 August 10, 2018 07:42AM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 187 August 27, 2018 05:40PM

Re: cache: move open to thread pool

Maxim Dounin 277 September 03, 2018 12:10PM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 202 September 04, 2018 08:00PM

Re: cache: move open to thread pool

Maxim Dounin 197 September 13, 2018 02:12PM

Re: cache: move open to thread pool

Roman Arutyunyan 262 October 04, 2018 05:34AM

Re: cache: move open to thread pool

Roman Arutyunyan 323 November 01, 2018 08:30AM

Re: cache: move open to thread pool

Maxim Konovalov 188 November 01, 2018 12:08PM

Re: cache: move open to thread pool

Maxim Konovalov 197 November 08, 2018 09:20AM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 182 November 15, 2018 05:54PM

Re: cache: move open to thread pool

Maxim Konovalov 194 November 16, 2018 04:12AM

Re: cache: move open to thread pool

Maxim Konovalov 194 December 05, 2018 04:44AM

Re: cache: move open to thread pool

Maxim Konovalov 149 January 10, 2019 03:32AM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 167 January 14, 2019 08:54PM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 132 January 22, 2019 01:22PM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 159 January 22, 2019 02:36PM

Re: cache: move open to thread pool

Maxim Konovalov 149 January 23, 2019 07:40AM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 134 February 06, 2019 04:20PM

Re: cache: move open to thread pool

Maxim Konovalov 142 February 07, 2019 08:40AM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 177 February 08, 2019 05:42PM

Re: cache: move open to thread pool

karton 74 July 19, 2019 11:28AM

Re: cache: move open to thread pool

Maxim Konovalov 82 July 19, 2019 02:44PM

Re: cache: move open to thread pool

karton 68 July 19, 2019 09:46PM

Re: cache: move open to thread pool

Roman Arutyunyan 67 July 22, 2019 10:06AM

Re: cache: move open to thread pool

karton 71 July 22, 2019 11:40AM

Re: cache: move open to thread pool

karton 79 July 23, 2019 11:48PM

Re: cache: move open to thread pool

Roman Arutyunyan 62 August 16, 2019 10:26AM

Re: cache: move open to thread pool

karton 81 August 16, 2019 10:52AM



Sorry, you do not have permission to post/reply in this forum.

Online Users

Guests: 82
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready