October 17, 2011 12:27AM
I have nginx with my custom module, that rewrite content on some conditions. Without it everything works fine, but after enabling it, nginx start to crash approx. every 2 hours ( > 100 req./sec) coredump shows that it crahses in gzip module:

Core was generated by `nginx: worker process '.
Program terminated with signal 11, Segmentation fault.
#0 0x000000343ea0286d in crc32 () from /usr/lib64/libz.so.1
(gdb) bt
#0 0x000000343ea0286d in crc32 () from /usr/lib64/libz.so.1
#1 0x0000000000450f3f in ngx_http_gzip_filter_add_data (r=0x601f380, in=0x29ac280) at src/http/modules/ngx_http_gzip_filter_module.c:708
#2 ngx_http_gzip_body_filter (r=0x601f380, in=0x29ac280) at src/http/modules/ngx_http_gzip_filter_module.c:394
#3 0x0000000000451ac5 in ngx_http_postpone_filter (r=0x601f380, in=0x29ac280) at src/http/ngx_http_postpone_filter_module.c:82
#4 0x00000000004521b1 in ngx_http_ssi_body_filter (r=0x343ea0c9c0, in=0x7906442d) at src/http/modules/ngx_http_ssi_filter_module.c:392
#5 0x00000000004564b5 in ngx_http_charset_body_filter (r=0x343ea0c9c0, in=0x61c0ff8) at src/http/modules/ngx_http_charset_filter_module.c:552
#6 0x0000000000457a5c in ngx_http_sub_body_filter (r=0x343ea0c9c0, in=0x29ac280) at src/http/modules/ngx_http_sub_filter_module.c:188
#7 0x0000000000470e8f in ngx_http_af_filter (r=0x601f380, in=0x29ac280) at /usr/src/redhat/SOURCES/af-headers/ngx_af_headers_module.c:768
#8 0x00000000004793e6 in clweb_c_body_filter (r=0x601f380, in=0x29ac280) at /usr/src/redhat/SOURCES/content-parser-module/ngx_mod_content_parser.c:510
#9 0x0000000000479d28 in ngx_http_gunzip_body_filter (r=0x601f380, in=0x29ac280) at /usr/src/redhat/SOURCES/gunzip/ngx_http_gunzip_filter_module.c:323
#10 0x000000000047ca5d in ngx_subr_body_filter (r=0x601f380, in=0x1f5f830) at /usr/src/redhat/SOURCES/ngx_subr_module/ngx_subr_module.c:219
#11 0x000000000047d46d in ngx_subr_body_filter (r=0x601f380, in=0x1f5f830) at /usr/src/redhat/SOURCES/ngx_subr_all_module/ngx_subr_all_module.c:219
#12 0x000000000040ba99 in ngx_output_chain (ctx=0x5e958c0, in=0x61c0ff8) at src/core/ngx_output_chain.c:65
#13 0x000000000043bbf7 in ngx_http_copy_filter (r=0x601f380, in=0x1f5f830) at src/http/ngx_http_copy_filter_module.c:141
#14 0x000000000044bdd1 in ngx_http_range_body_filter (r=0x343ea0c9c0, in=0x61c0ff8) at src/http/modules/ngx_http_range_filter_module.c:551
#15 0x000000000042ed92 in ngx_http_output_filter (r=0x601f380, in=0x1f5f830) at src/http/ngx_http_core_module.c:1868
#16 0x00000000004463fa in ngx_http_upstream_process_non_buffered_request (r=0x601f380, do_write=<value optimized out>) at src/http/ngx_http_upstream.c:2381
#17 0x00000000004468fc in ngx_http_upstream_process_non_buffered_upstream (r=0x601f380, u=0x6096858) at src/http/ngx_http_upstream.c:2352
#18 0x00000000004457f6 in ngx_http_upstream_handler (ev=0x148e568) at src/http/ngx_http_upstream.c:917
#19 0x00000000004269be in ngx_epoll_process_events (cycle=0x796480, timer=<value optimized out>, flags=<value optimized out>) at src/event/modules/ngx_epoll_module.c:635
#20 0x000000000041e3c8 in ngx_process_events_and_timers (cycle=0x796480) at src/event/ngx_event.c:245
#21 0x0000000000425203 in ngx_worker_process_cycle (cycle=0x796480, data=<value optimized out>) at src/os/unix/ngx_process_cycle.c:800
#22 0x0000000000423967 in ngx_spawn_process (cycle=0x796480, proc=0x425118 <ngx_worker_process_cycle>, data=0x0, name=0x4d01ce "worker process", respawn=-3) at src/os/unix/ngx_process.c:196
#23 0x000000000042478c in ngx_start_worker_processes (cycle=0x796480, n=12, type=-3) at src/os/unix/ngx_process_cycle.c:360
#24 0x0000000000425964 in ngx_master_process_cycle (cycle=0x796480) at src/os/unix/ngx_process_cycle.c:136
#25 0x0000000000408dba in main (argc=22, argv=0x795060) at src/core/nginx.c:405

dmesg says:
nginx[20500]: segfault at 61c1000 ip 000000343ea0286d sp 00007fff348eec78 error 4 in libz.so.1.2.3[343ea00000+14000]

so libz is going to read smth at bad address.
This is definetly not nginx bug, but it is not simple to debug nginx, so I will be happy for any advice, what can cause nginx to crash? May be its stack problems?

Server info: Centos linux 5.6, amd64, nginx 1.0.6, zlib-devel-1.2.3-3
Subject Author Posted

nginx with custom modules crashes in gzip crc32()

artemg October 17, 2011 12:27AM

Re: nginx with custom modules crashes in gzip crc32()

Igor Sysoev October 17, 2011 04:28AM

Re: nginx with custom modules crashes in gzip crc32()

artemg October 18, 2011 08:20PM

Re: nginx with custom modules crashes in gzip crc32()

agentzh October 18, 2011 11:26PM

Re: nginx with custom modules crashes in gzip crc32()

artemg October 19, 2011 04:19AM

Re: nginx with custom modules crashes in gzip crc32()

artemg October 20, 2011 07:17AM

Re: nginx with custom modules crashes in gzip crc32()

agentzh October 20, 2011 07:54AM

Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 134
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready