Welcome! Log In Create A New Profile

Advanced

[PATCH 18 of 31] Memcached: memcached_gzip_flag directive

Maxim Dounin
June 27, 2011 01:12PM
# HG changeset patch
# User Maxim Dounin <mdounin@mdounin.ru>
# Date 1309187038 -14400
# Node ID e56caa64771c6fcffa661943b8b3e4f2a291d485
# Parent 31aaaef96cb108e118afe407346cb4f04a444437
Memcached: memcached_gzip_flag directive.

This directive allows to test desired flag as returned by memcached and
sets Content-Encoding to gzip if one found.

This is reimplementation of patch by Tomash Brechko as available on
http://openhack.ru/. It should be a bit more correct though (at least
I think so). In particular, it doesn't try to detect if we are able to
gunzip data, but instead just sets correct Content-Encoding.

Originally by: Tomash Brechko

diff --git a/src/http/modules/ngx_http_memcached_module.c b/src/http/modules/ngx_http_memcached_module.c
--- a/src/http/modules/ngx_http_memcached_module.c
+++ b/src/http/modules/ngx_http_memcached_module.c
@@ -12,6 +12,7 @@
typedef struct {
ngx_http_upstream_conf_t upstream;
ngx_int_t index;
+ ngx_uint_t gzip_flag;
} ngx_http_memcached_loc_conf_t;


@@ -100,6 +101,13 @@ static ngx_command_t ngx_http_memcached
offsetof(ngx_http_memcached_loc_conf_t, upstream.next_upstream),
&ngx_http_memcached_next_upstream_masks },

+ { ngx_string("memcached_gzip_flag"),
+ NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
+ ngx_conf_set_num_slot,
+ NGX_HTTP_LOC_CONF_OFFSET,
+ offsetof(ngx_http_memcached_loc_conf_t, gzip_flag),
+ NULL },
+
ngx_null_command
};

@@ -280,10 +288,13 @@ ngx_http_memcached_reinit_request(ngx_ht
static ngx_int_t
ngx_http_memcached_process_header(ngx_http_request_t *r)
{
- u_char *p, *len;
- ngx_str_t line;
- ngx_http_upstream_t *u;
- ngx_http_memcached_ctx_t *ctx;
+ u_char *p, *start;
+ ngx_str_t line;
+ ngx_uint_t flags;
+ ngx_table_elt_t *h;
+ ngx_http_upstream_t *u;
+ ngx_http_memcached_ctx_t *ctx;
+ ngx_http_memcached_loc_conf_t *mlcf;

u = r->upstream;

@@ -308,6 +319,7 @@ found:
p = u->buffer.pos;

ctx = ngx_http_get_module_ctx(r, ngx_http_memcached_module);
+ mlcf = ngx_http_get_module_loc_conf(r, ngx_http_memcached_module);

if (ngx_strncmp(p, "VALUE ", sizeof("VALUE ") - 1) == 0) {

@@ -328,23 +340,56 @@ found:
goto no_valid;
}

- /* skip flags */
+ /* flags */
+
+ start = p;

while (*p) {
if (*p++ == ' ') {
- goto length;
+ if (mlcf->gzip_flag) {
+ goto flags;
+ } else {
+ goto length;
+ }
}
}

goto no_valid;

+ flags:
+
+ flags = ngx_atoi(start, p - start - 1);
+
+ if (flags == (ngx_uint_t) NGX_ERROR) {
+ ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+ "memcached sent invalid flags in response \"%V\" "
+ "for key \"%V\"",
+ &line, &ctx->key);
+ return NGX_HTTP_UPSTREAM_INVALID_HEADER;
+ }
+
+ if (flags & mlcf->gzip_flag) {
+ h = ngx_list_push(&r->headers_out.headers);
+ if (h == NULL) {
+ return NGX_ERROR;
+ }
+
+ h->hash = 1;
+ h->key.len = sizeof("Content-Encoding") - 1;
+ h->key.data = (u_char *) "Content-Encoding";
+ h->value.len = sizeof("gzip") - 1;
+ h->value.data = (u_char *) "gzip";
+
+ r->headers_out.content_encoding = h;
+ }
+
length:

- len = p;
+ start = p;

while (*p && *p++ != CR) { /* void */ }

- r->headers_out.content_length_n = ngx_atoof(len, p - len - 1);
+ r->headers_out.content_length_n = ngx_atoof(start, p - start - 1);
if (r->headers_out.content_length_n == -1) {
ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
"memcached sent invalid length in response \"%V\" "
@@ -533,6 +578,7 @@ ngx_http_memcached_create_loc_conf(ngx_c
conf->upstream.pass_request_body = 0;

conf->index = NGX_CONF_UNSET;
+ conf->gzip_flag = NGX_CONF_UNSET_UINT;

return conf;
}
@@ -576,6 +622,8 @@ ngx_http_memcached_merge_loc_conf(ngx_co
conf->index = prev->index;
}

+ ngx_conf_merge_uint_value(conf->gzip_flag, prev->gzip_flag, 0);
+
return NGX_CONF_OK;
}


_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://nginx.org/mailman/listinfo/nginx-devel
Subject Author Views Posted

[PATCH 00 of 31] generic patch queue for 1.0.4

Maxim Dounin 3866 June 27, 2011 01:10PM

[PATCH 01 of 31] Cache: fix another "stalled cache updating" alert

Maxim Dounin 1021 June 27, 2011 01:10PM

Re: [PATCH 01 of 31] Cache: fix another "stalled cache updating" alert

Kirill A. Korinskiy 995 June 27, 2011 02:06PM

Re: [PATCH 01 of 31] Cache: fix another "stalled cache updating" alert

Kirill A. Korinskiy 1011 June 27, 2011 03:00PM

[PATCH 02 of 31] Fastcgi: fix fastcgi_param with "HTTP_"

Maxim Dounin 1037 June 27, 2011 01:10PM

[PATCH 03 of 31] Bugfix: https wasn't working on systems with 32-bit off_t

Maxim Dounin 1177 June 27, 2011 01:10PM

[PATCH 04 of 31] Upstream: fix request finalization if client timed out

Maxim Dounin 949 June 27, 2011 01:10PM

[PATCH 05 of 31] Upstream: properly allocate memory for tried flags

Maxim Dounin 1045 June 27, 2011 01:10PM

[PATCH 06 of 31] Complain on invalid log levels

Maxim Dounin 1274 June 27, 2011 01:10PM

[PATCH 07 of 31] Fix incorrect 201 replies from dav module

Maxim Dounin 1046 June 27, 2011 01:10PM

[PATCH 08 of 31] Fix double content when return is used in error_page redirection

Maxim Dounin 1118 June 27, 2011 01:10PM

[PATCH 09 of 31] Drop incorrect special case for return 204

Maxim Dounin 1079 June 27, 2011 01:10PM

[PATCH 10 of 31] Clear old Location header (if any) while adding new one

Maxim Dounin 1051 June 27, 2011 01:10PM

[PATCH 11 of 31] Better handle various per-server ssl options with SNI

Maxim Dounin 1197 June 27, 2011 01:10PM

[PATCH 12 of 31] Better handle late upstream creation

Maxim Dounin 964 June 27, 2011 01:12PM

[PATCH 13 of 31] Gzip filter: handle empty flush buffers

Maxim Dounin 1120 June 27, 2011 01:12PM

[PATCH 14 of 31] Fix connection drops with AIO

Maxim Dounin 904 June 27, 2011 01:12PM

[PATCH 15 of 31] Fix socket leak with "aio sendfile" and "limit_rate" directives

Maxim Dounin 1122 June 27, 2011 01:12PM

[PATCH 16 of 31] Correctly handle Content-Encoding set from perl

Maxim Dounin 901 June 27, 2011 01:12PM

[PATCH 17 of 31] Gzip static: "always" parameter in "gzip_static" directive

Maxim Dounin 1087 June 27, 2011 01:12PM

Re: [PATCH 17 of 31] Gzip static: "always" parameter in "gzip_static" directive

Zhu Qun-Ying 958 June 27, 2011 02:02PM

Re: [PATCH 17 of 31] Gzip static: "always" parameter in "gzip_static" directive

Maxim Dounin 1063 June 28, 2011 06:34AM

[PATCH 18 of 31] Memcached: memcached_gzip_flag directive

Maxim Dounin 1014 June 27, 2011 01:12PM

[PATCH 19 of 31] Mail: handle smtp multiline replies

Maxim Dounin 1001 June 27, 2011 01:12PM

[PATCH 20 of 31] Additional headers for proxy_ignore_headers/fastcgi_ignore_headers

Maxim Dounin 1120 June 27, 2011 01:12PM

[PATCH 21 of 31] Fix cpu hog with all upstream servers marked "down"

Maxim Dounin 950 June 27, 2011 01:12PM

Re: [PATCH 21 of 31] Fix cpu hog with all upstream servers marked "down"

Oded Arbel 983 August 15, 2011 11:00AM

Re: [PATCH 21 of 31] Fix cpu hog with all upstream servers marked "down"

Maxim Dounin 983 August 15, 2011 12:00PM

Re: [PATCH 21 of 31] Fix cpu hog with all upstream servers marked "down"

lanshun zhou 928 August 15, 2011 01:52PM

Re: [PATCH 21 of 31] Fix cpu hog with all upstream servers marked "down"

Oded Arbel 1011 August 15, 2011 11:10AM

Re: [PATCH 21 of 31] Fix cpu hog with all upstream servers marked "down"

Oded Arbel 1262 August 15, 2011 02:46PM

[PATCH 22 of 31] Cache: correctly set conf_file while adding paths

Maxim Dounin 1167 June 27, 2011 01:12PM

[PATCH 23 of 31] Upstream: fix proxy_store leaving temporary files for subrequests

Maxim Dounin 1265 June 27, 2011 01:12PM

[PATCH 24 of 31] Cache: fix sending of empty responses

Maxim Dounin 1037 June 27, 2011 01:14PM

[PATCH 25 of 31] Cache: fix sending of stale responses

Maxim Dounin 1162 June 27, 2011 01:14PM

[PATCH 26 of 31] Variables: honor no_cacheable for not_found variables

Maxim Dounin 1132 June 27, 2011 01:14PM

[PATCH 27 of 31] Core: protect from subrequest loops

Maxim Dounin 1053 June 27, 2011 01:14PM

[PATCH 28 of 31] Core: resolve various cycles with named locations and post_action

Maxim Dounin 1092 June 27, 2011 01:14PM

[PATCH 29 of 31] Autoindex: escape '?' in file names

Maxim Dounin 981 June 27, 2011 01:14PM

[PATCH 30 of 31] Autoindex: escape html in file names

Maxim Dounin 868 June 27, 2011 01:14PM

[PATCH 31 of 31] Unbreak build with embedded perl and --with-openssl

Maxim Dounin 904 June 27, 2011 01:14PM

Re: [PATCH 00 of 31] generic patch queue for 1.0.4

António P. P. Almeida 943 June 27, 2011 10:10PM

Re: [PATCH 00 of 31] generic patch queue for 1.0.4

Maxim Dounin 1126 June 28, 2011 10:40AM

Re: [PATCH 00 of 31] generic patch queue for 1.0.4

fanboy 976 June 28, 2011 01:48AM

Re: [PATCH 00 of 31] generic patch queue for 1.0.4

Maxim Dounin 1195 June 28, 2011 11:00AM



Sorry, you do not have permission to post/reply in this forum.

Online Users

Guests: 233
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready