Welcome! Log In Create A New Profile

Advanced

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel
January 22, 2019 02:36PM
I spoke too soon, just realized our test colo is running the patches
without aio_open on. Flipping that switch now.

Also, including the patch that we applied on top in addition to the
massaging we did to resolve conflicts. I haven't dug too deep to see
if stock nginx also requires similar changes or they are only
necessary because of our other nginx changes:

--- src/http/ngx_http_file_cache.c
+++ src/http/ngx_http_file_cache.c
@@ -611,6 +611,12 @@ ngx_http_do_file_cache_open(ngx_http_request_t
*r, unsigned ignore_vary)
return NGX_ERROR;
}

+ c->buf = ngx_create_temp_buf(r->pool, c->body_start);
+ if (c->buf == NULL) {
+ return NGX_ERROR;
+ }
+ c->file.fd = NGX_INVALID_FILE;
+
if (!test) {
/*
* Either the request is cacheable but the file doesn't
@@ -1148,6 +1154,7 @@ ngx_http_file_cache_aio_open(ngx_http_request_t
*r, ngx_http_cache_t *c)
ngx_memzero(&r->open_file_info, sizeof(ngx_open_file_info_t));

r->open_file_info.uniq = c->uniq;
+ r->open_file_info.fd = NGX_INVALID_FILE;
r->open_file_info.valid = clcf->open_file_cache_valid;
r->open_file_info.min_uses = clcf->open_file_cache_min_uses;
r->open_file_info.events = clcf->open_file_cache_events;
@@ -1205,11 +1212,6 @@ ngx_http_file_cache_aio_open(ngx_http_request_t
*r, ngx_http_cache_t *c)
c->length = r->open_file_info.size;
c->fs_size = (r->open_file_info.fs_size + cache->bsize - 1) / cache->bsize;

- c->buf = ngx_create_temp_buf(r->pool, c->body_start);
- if (c->buf == NULL) {
- return NGX_ERROR;
- }
-
return NGX_OK;
}

On Tue, Jan 22, 2019 at 10:20 AM Ka-Hing Cheung <kahing@cloudflare.com> wrote:
>
> Hi Maxim and Roman,
>
> We've been running a version of your patches in our test colo for the
> past week with no ill effects, with the caveat that it's difficult to
> measure performance impacts in our test colo because of varying
> traffic. We had to massage the patches a bit because we don't run
> vanilla nginx. Also we've only tried with open file cache disabled.
>
> Let me know if you are looking for other feedback.
>
> - Ka-Hing
>
> On Mon, Jan 14, 2019 at 5:53 PM Ka-Hing Cheung <kahing@cloudflare.com> wrote:
> >
> > We didn't get to it last year because of other priorities but this is
> > being worked on. We don't use a vanilla nginx so it takes effort to
> > integrate your patches. Having a version of this in upstream nginx is
> > important to us and to me personally.
> >
> > On Thu, Jan 10, 2019 at 12:30 AM Maxim Konovalov <maxim@nginx.com> wrote:
> > >
> > > So guys, are you really interested in this work beyond shiny
> > > marketing blog posts?
> > >
> > > On 05/12/2018 12:43, Maxim Konovalov wrote:
> > > > Hello,
> > > >
> > > > just a reminder that we are looking for a tester for these patches.
> > > >
> > > > Thanks,
> > > >
> > > > Maxim
> > > >
> > > > On 16/11/2018 12:10, Maxim Konovalov wrote:
> > > >> Thanks, Ka-Hing, we'll wait for your feedback.
> > > >>
> > > >> On 16/11/2018 01:53, Ka-Hing Cheung wrote:
> > > >>> Hi,
> > > >>>
> > > >>> I didn't forget about this. I am pretty swamped at the moment and
> > > >>> there's a holiday freeze coming up. Will get to his in December.
> > > >>>
> > > >>> - Ka-Hing
> > > >>>
> > > >>> On Thu, Nov 8, 2018 at 6:19 AM Maxim Konovalov <maxim@nginx.com> wrote:
> > > >>>>
> > > >>>> Hi Ka-Hing,
> > > >>>>
> > > >>>> would you mind to test Roman's most recent patches that add
> > > >>>> "aio_open" directive?
> > > >>>>
> > > >>>> http://mailman.nginx.org/pipermail/nginx-devel/2018-November/011538.html
> > > >>>>
> > > >>>> We are looking for overall performance and stability metrics and
> > > >>>> feedback.
> > > >>>>
> > > >>>> Much appreciated,
> > > >>>>
> > > >>>> Maxim
> > > >>>>
> > > >>>> --
> > > >>>> Maxim Konovalov
> > > >>
> > > >>
> > > >
> > > >
> > >
> > >
> > > --
> > > Maxim Konovalov
> > > _______________________________________________
> > > nginx-devel mailing list
> > > nginx-devel@nginx.org
> > > http://mailman.nginx.org/mailman/listinfo/nginx-devel
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel
Subject Author Views Posted

cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 2816 August 07, 2018 05:28PM

Re: cache: move open to thread pool

Maxim Dounin 840 August 08, 2018 02:18PM

RE: cache: move open to thread pool

erankor 798 August 08, 2018 03:46PM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 626 August 27, 2018 06:00PM

RE: cache: move open to thread pool

erankor 733 August 28, 2018 05:18AM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 628 August 09, 2018 05:44PM

Re: cache: move open to thread pool

Maxim Dounin 1091 August 10, 2018 07:42AM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 635 August 27, 2018 05:40PM

Re: cache: move open to thread pool

Maxim Dounin 751 September 03, 2018 12:10PM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 682 September 04, 2018 08:00PM

Re: cache: move open to thread pool

Maxim Dounin 642 September 13, 2018 02:12PM

Re: cache: move open to thread pool

Roman Arutyunyan 754 October 04, 2018 05:34AM

Re: cache: move open to thread pool

Roman Arutyunyan 743 November 01, 2018 08:30AM

Re: cache: move open to thread pool

Maxim Konovalov 625 November 01, 2018 12:08PM

Re: cache: move open to thread pool

Roman Arutyunyan 459 August 03, 2021 04:52AM

Re: cache: move open to thread pool

Maxim Konovalov 624 November 08, 2018 09:20AM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 662 November 15, 2018 05:54PM

Re: cache: move open to thread pool

Maxim Konovalov 696 November 16, 2018 04:12AM

Re: cache: move open to thread pool

Maxim Konovalov 666 December 05, 2018 04:44AM

Re: cache: move open to thread pool

Maxim Konovalov 609 January 10, 2019 03:32AM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 661 January 14, 2019 08:54PM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 579 January 22, 2019 01:22PM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 655 January 22, 2019 02:36PM

Re: cache: move open to thread pool

Maxim Konovalov 620 January 23, 2019 07:40AM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 602 February 06, 2019 04:20PM

Re: cache: move open to thread pool

Maxim Konovalov 644 February 07, 2019 08:40AM

Re: cache: move open to thread pool

Ka-Hing Cheung via nginx-devel 690 February 08, 2019 05:42PM

Re: cache: move open to thread pool

karton 524 July 19, 2019 11:28AM

Re: cache: move open to thread pool

Maxim Konovalov 567 July 19, 2019 02:44PM

Re: cache: move open to thread pool

karton 493 July 19, 2019 09:46PM

Re: cache: move open to thread pool

Roman Arutyunyan 542 July 22, 2019 10:06AM

Re: cache: move open to thread pool

karton 517 July 22, 2019 11:40AM

Re: cache: move open to thread pool

karton 535 July 23, 2019 11:48PM

Re: cache: move open to thread pool

Roman Arutyunyan 601 August 16, 2019 10:26AM

Re: cache: move open to thread pool

karton 657 August 16, 2019 10:52AM

Re: cache: move open to thread pool

NCviQwilt 511 September 02, 2021 12:22PM

Re: cache: move open to thread pool

Roman Arutyunyan 556 September 09, 2021 06:50AM



Sorry, you do not have permission to post/reply in this forum.

Online Users

Guests: 128
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready