Theres actually an example using the memcached module (and echo module to
get the request body). The main advantage to the lua module is if you
wanted to do something custom (e.g processing) or transmit it over a
different protocol not available as an extension.


On Mon, May 26, 2014 at 7:17 PM, Paulo Silva <pauloasilva@gmail.com> wrote:

> On Mon, May 26, 2014 at 9:28 AM, SplitIce <mat999@gmail.com> wrote:
> > Yes, connecting with SOCK_NONBLOCK shouldnt block. I don't believe this
> is
> > mentioned previously. If your code blocks (e.g blocking connect or
> blocking
> > send) then it would reduce nginx throughput substantially. Thats the
> point I
> > was trying to make.
> >
>
> Yap, I didn't mention SOCK_NONBLOCK. Sorry about that.
>
> Although I'm working on a prototype to benchmark, my concern is about
> socket() and connect() calls on every nginx filter execution. How
> would you manage this?
>
> > Have you investigated srcache
> > (https://github.com/openresty/srcache-nginx-module/)? srcache_store
> could be
> > used without srcache_fetch to store only. Your store handler could be a
> > proxy_pass call to pass a restful server or whatever you use for storing
> the
> > HTML responses.
> >
>
> I didn't know about this module. Thanks, I will have a deep look on it.
> Do you think that I can pick the proxy_pass response and store it on
> memcache easily?
>
> > Perhaps a better solution would be to use the nginx lua extension instead
> > (less of a hack).
> >
>
> Do you think that lua extension has advantages regarding an nginx filter
> module.
>
> Sorry about so many questions but at the beginning, nginx is a all
> mystery, and you'll worried to do not break its performance.
>
> Thanks a bunch,
>
> > Regards,
> > Mathew
> >
> >
> > On Mon, May 26, 2014 at 6:18 PM, Paulo Silva <pauloasilva@gmail.com>
> wrote:
> >>
> >> On Mon, May 26, 2014 at 9:09 AM, SplitIce <mat999@gmail.com> wrote:
> >> > As in blocking send and connect? I don't know the specifics of Unix
> >> > Sockets,
> >> > but don't they block when the buffer fills (I know FIFO queues do)?
> >> >
> >>
> >> Sorry, I don't fully understand your question.
> >> I was expecting that with the SOCK_NONBLOCK it would not block.
> >>
> >> What would be your approach?
> >> Do you know about any nginx internal mechanism to accomplish this goal
> >> (get the upstream response body out of nginx)?
> >>
> >> >
> >> > On Mon, May 26, 2014 at 9:22 AM, Paulo Silva <pauloasilva@gmail.com>
> >> > wrote:
> >> >>
> >> >> Hi,
> >> >> I'm not sure whether I will face problems with other filters
> modifying
> >> >> the response body after mine, but for know I'm comfortable as I can
> >> >> rebuild the full response body just iterating buffers chains.
> >> >>
> >> >> As I said before I'm using nginx as reverse proxy and my main goal is
> >> >> to pass the upstream (proxy_pass) response to another local process
> >> >> (relative to nginx).
> >> >>
> >> >> I am benchmarking Unix sockets and Shared memory as IPC.
> >> >> I did it already for Unix Sockets and with my prototype the nginx
> >> >> "performance" dropped for half the number of requests per second. Of
> >> >> course I'm doing something really bad.
> >> >>
> >> >> Is it OK to use socket/connect/send from inside an nginx module?
> >> >>
> >> >> I would be glad to hear from you.
> >> >> Thanks,
> >> >>
> >> >> On Fri, May 23, 2014 at 2:50 PM, Paulo Silva <pauloasilva@gmail.com>
> >> >> wrote:
> >> >> > Because I don't have deep knowledge of nginx internal and I can not
> >> >> > find a proper resource about it, the best I can do and with what I
> am
> >> >> > comfortable is with body_filter.
> >> >> >
> >> >> > Do you think I can notice whether all other 3rd party module
> filters
> >> >> > finish modifying the ngx_chain_t *in ?
> >> >> >
> >> >> >
> >> >> >
> >> >> > On Fri, May 23, 2014 at 2:41 PM, Maxim Dounin <mdounin@mdounin.ru>
> >> >> > wrote:
> >> >> >> Hello!
> >> >> >>
> >> >> >> On Fri, May 23, 2014 at 02:17:27PM +0100, Paulo Silva wrote:
> >> >> >>
> >> >> >>> Hi,
> >> >> >>>
> >> >> >>> On Fri, May 23, 2014 at 12:58 PM, Maxim Dounin <
> mdounin@mdounin.ru>
> >> >> >>> wrote:
> >> >> >>> > Hello!
> >> >> >>> >
> >> >> >>> > On Fri, May 23, 2014 at 11:57:20AM +0100, Paulo Silva wrote:
> >> >> >>> >
> >> >> >>> >> there is other option than modify the auto/modules file?
> >> >> >>> >>
> >> >> >>> >> According to my goal (capture the full request response body)
> I
> >> >> >>> >> would
> >> >> >>> >> say that my module must run right before the postpone.
> >> >> >>> >
> >> >> >>> > Before the postpone filter you'll get subrequest bodies in your
> >> >> >>> > filter, which is probably not what you want (the postpone
> filter
> >> >> >>> > is to glue subrequest together, correctly ordered).
> >> >> >>> >
> >> >> >>> >> Am I supposed to modify the auto/modules like follows?
> >> >> >>> >>
> >> >> >>> >> if [ $HTTP_POSTPONE = YES ]; then
> >> >> >>> >> HTTP_FILTER_MODULES="$HTTP_FILTER_MODULES
> >> >> >>> >> $HTTP_POSTPONE_FILTER_MODULE"
> >> >> >>> >> HTTP_SRCS="$HTTP_SRCS $HTTP_POSTPONE_FILTER_SRCS"
> >> >> >>> >> fi
> >> >> >>> >>
> >> >> >>> >> # insert my module here!
> >> >> >>> >>
> >> >> >>> >> if [ $HTTP_SSI = YES ]; then
> >> >> >>> >> have=NGX_HTTP_SSI . auto/have
> >> >> >>> >> HTTP_FILTER_MODULES="$HTTP_FILTER_MODULES
> >> >> >>> >> $HTTP_SSI_FILTER_MODULE"
> >> >> >>> >> HTTP_DEPS="$HTTP_DEPS $HTTP_SSI_DEPS"
> >> >> >>> >> HTTP_SRCS="$HTTP_SRCS $HTTP_SSI_SRCS"
> >> >> >>> >> fi
> >> >> >>> >>
> >> >> >>> >>
> >> >> >>> >> I did check my modules config file and I did realize that it
> is
> >> >> >>> >> "queued" as HTTP_AUX_FILTER_MODULES. There are different
> queues
> >> >> >>> >> for
> >> >> >>> >> core modules and addons?
> >> >> >>> >
> >> >> >>> > The HTTP_AUX_FILTER_MODULES is a generic queue, and it's the
> >> >> >>> > only one currently officially supported for 3rd party modules.
> >> >> >>> >
> >> >> >>> > If you want your filter to be called right before/after
> postpone
> >> >> >>> > filter, it should be relatively safe to put it into the
> >> >> >>> > HTTP_POSTPONE_FILTER_MODULE variable though (and may be with
> some
> >> >> >>> > additional checks to make sure the postpone filter is enabled,
> or
> >> >> >>> > just a code to enable it unconditionally).
> >> >> >>> >
> >> >> >>>
> >> >> >>> And this is also valid when compiling nginx with the --add-module
> >> >> >>> flag?
> >> >> >>> How does config file look like?
> >> >> >>>
> >> >> >>> My knowledge is restricted to Emiller's Guide To Nginx Module
> >> >> >>> Development (http://www.evanmiller.org/nginx-modules-guide.html)
> >> >> >>> and a
> >> >> >>> few debugging hours.
> >> >> >>
> >> >> >> Uhm, looking again into auto/modules I think I was wrong, and
> >> >> >> modifying the HTTP_POSTPONE_FILTER_MODULE variable won't work
> >> >> >> (added module config scripts are executed later on), you should
> >> >> >> modify HTTP_FILTER_MODULES variable instead, and put your module
> >> >> >> into a proper position.
> >> >> >>
> >> >> >> Note that the "config" file of a module is just a shell script,
> >> >> >> and you are free to do more or less anything there.
> >> >> >>
> >> >> >> --
> >> >> >> Maxim Dounin
> >> >> >> http://nginx.org/
> >> >> >>
> >> >> >> _______________________________________________
> >> >> >> nginx-devel mailing list
> >> >> >> nginx-devel@nginx.org
> >> >> >> http://mailman.nginx.org/mailman/listinfo/nginx-devel
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Paulo A. Silva
> >> >> > http://tech.pauloasilva.com
> >> >> > http://linkedin.com/in/devpauloasilva/
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Paulo A. Silva
> >> >> http://tech.pauloasilva.com
> >> >> http://linkedin.com/in/devpauloasilva/
> >> >>
> >> >> _______________________________________________
> >> >> nginx-devel mailing list
> >> >> nginx-devel@nginx.org
> >> >> http://mailman.nginx.org/mailman/listinfo/nginx-devel
> >> >
> >> >
> >> >
> >> > _______________________________________________
> >> > nginx-devel mailing list
> >> > nginx-devel@nginx.org
> >> > http://mailman.nginx.org/mailman/listinfo/nginx-devel
> >>
> >>
> >>
> >> --
> >> Paulo A. Silva
> >> http://tech.pauloasilva.com
> >> http://linkedin.com/in/devpauloasilva/
> >>
> >> _______________________________________________
> >> nginx-devel mailing list
> >> nginx-devel@nginx.org
> >> http://mailman.nginx.org/mailman/listinfo/nginx-devel
> >
> >
> >
> > _______________________________________________
> > nginx-devel mailing list
> > nginx-devel@nginx.org
> > http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
>
>
> --
> Paulo A. Silva
> http://tech.pauloasilva.com
> http://linkedin.com/in/devpauloasilva/
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel