Welcome! Log In Create A New Profile

Advanced

[ANN] ngx_lua v0.1.5: ability to capture multiple parallel subrequests

agentzh
February 09, 2011 05:08AM
Hi, folks!

I'm pleased to announce that ngx_lua v0.1.5 is now released. You can
download the release tarball from the download page:

    https://github.com/chaoslawful/lua-nginx-module/downloads

This version of the ngx_lua module implements the
ngx.location.capture_multi method that can issue multiple nginx
subrequests at the same time, capture their responses along the way,
and then return the results in the same order that they're specified
in the arguments.

Here is a small example:

    location ~ '^/echo/(\w+)$' {
        echo -n $1;
    }

    location = /main {
        content_by_lua '
            local res1, res2, res3 =
                ngx.location.capture_multi{
                    { "/echo/sun" },
                    { "/echo/moon" },
                    { "/echo/earth" },
                }
            ngx.say(res1.body)
            ngx.say(res2.body)
            ngx.say(res3.body)
        ';
    }

then accessing /main gives

    sun
    moon
    earth

and those three subrequests, GET /echo/sun, GET /echo/moon, and GET
/echo/earth, were issued at the same time.

Below is a more complicated example that fires 2 concurrent MySQL
queries to calculate the count of dogs and the count cats,
respectively, at the same time, and finally output the sum of these 2
counts, i.e., the total count of all the animals.

First of all, we define the remote mysql server upstream in the main http block:

    upstream mysql_node {
        drizzle_server 10.32.136.5:3306
            user=monty password=some_pass dbname=main protocol=mysql;
        drizzle_keepalive max=100 mode=single overflow=ignore;
    }

then we define a general-purpose internal location to proxy SQL
queries to remote MySQL node in our default virtual host server block:

    location = /mysql {
        internal;
        drizzle_query $echo_request_body;
        drizzle_pass mysql_node;
        rds_json on;
    }

after that, we define our main location that does the job:

    location /main {
        content_by_lua '
            local opts1 = {
                method = ngx.HTTP_POST,
                body = "select sum(1) cnt from cats",
            }
            local opts2 = {
                method = ngx.HTTP_POST,
                body = "select sum(1) cnt from dogs",
            }
            local res1, res2 =
                ngx.location.capture_multi{
                    { "/mysql", opts1 },
                    { "/mysql", opts2 }
                }

            local yajl = require("yajl")
            local cats = yajl.to_value(res1.body)
            local dogs = yajl.to_value(res2.body)

            local total = cats[1].cnt + dogs[1].cnt
            ngx.say("for total ", total, " animals.")
        ';
    }

For brevity, I've omitted all the error handling code like checking if
res1.status is 200 and validating cats.errcode has nil values ;)

The nginx modules ngx_drizzle, ngx_rds_json, and the lua-yajl Lua
library are also used in the sample above.

This example can be trivially extended to run tens or even more sql
queries to different MySQL machines and combined with more PostgreSQL
database servers. All in parallel and all are 100% non-blocking on
network traffic.

This v0.1.5 version of ngx_lua also contains

* optimizations on subrequest buffer management and thus saving
much memory when using ngx.location.capture (and
ngx.location.capture_multi).

* fixes of the build system that we no longer require OpenSSL and
ngx_lua no longer crash when used with statically linked OpenSSL.
Thanks Marcus Clyne and Vladislav Manchev.

* fixes of the Lua code cache handling that we no longer clear
pre-loaded standard Lua packages like "table" and "string" when the
Lua code cache is off.

The ngx_lua module embeds the Lua/LuaJIT interpreter into the nginx
core and integrates the powerful Lua threads (aka Lua coroutines) into
the nginx event model by means of nginx subrequests.

You can get the latest source code and full documentation from the
ngx_lua's project page:

   https://github.com/chaoslawful/lua-nginx-module/

Happy Lua and nginx.conf hacking!

Cheers,
-agentzh

_______________________________________________
nginx mailing list
nginx@nginx.org
http://nginx.org/mailman/listinfo/nginx
Subject Author Posted

[ANN] ngx_lua v0.1.5: ability to capture multiple parallel subrequests

agentzh February 09, 2011 05:08AM

Re: [ANN] ngx_lua v0.1.5: ability to capture multiple parallel subrequests

agentzh February 10, 2011 12:34AM

Re: [ANN] ngx_lua v0.1.5: ability to capture multiple parallel subrequests

Eugaia February 10, 2011 04:08AM

Re: [ANN] ngx_lua v0.1.5: ability to capture multiple parallel subrequests

Alexander Kunz February 10, 2011 05:28AM

Re: [ANN] ngx_lua v0.1.5: ability to capture multiple parallel subrequests

agentzh February 10, 2011 05:38AM

Re: [ANN] ngx_lua v0.1.5: ability to capture multiple parallel subrequests

Roman Vasilyev February 10, 2011 05:42PM

Re: [ANN] ngx_lua v0.1.5: ability to capture multiple parallel subrequests

Roman Vasilyev February 10, 2011 05:44PM

Re: [ANN] ngx_lua v0.1.5: ability to capture multiple parallel subrequests

kindy February 10, 2011 09:06PM

Re: [ANN] ngx_lua v0.1.5: ability to capture multiple parallel subrequests

agentzh February 11, 2011 12:06AM

Re: [ANN] ngx_lua v0.1.5: ability to capture multiple parallel subrequests

Roman Vasilyev February 11, 2011 03:00PM

Re: [ANN] ngx_lua v0.1.5: ability to capture multiple parallel subrequests

agentzh August 09, 2011 12:04AM

Re: [ANN] ngx_lua v0.1.5: ability to capture multiple parallel subrequests

agentzh August 09, 2011 12:06AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 195
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready