Welcome! Log In Create A New Profile

Advanced

Seg fault in http read event handler caused by rouge event call without context

Dave Brennan
November 18, 2019 09:04AM
For the last few years we have been using the "nginx_upload" module to streamline result posting within our environment.

With the introduction of nginx 1.17.5 we saw a large number of segment faults, causing us to revert to 1.17.4 on our development system.

While isolating the fault we added an increase in debug messages to monitor the request and context variables being passed to event handlers.

So a good response in 1.17.4 looks like this:-

2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload handle pre alloc Request address = 0000563E9FE451F0 Context = 0000000000000000
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload Handler post alloc Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload_eval_path Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload eval state path Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload client read Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 do read upload client Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 process request body Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 upload md5 variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload File size variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload Body Handler Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8

In 1.17.5 the event stream looks like this:-

2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload handle pre alloc Request address = 0000558ADDD4F780 Context = 0000000000000000
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload Handler post alloc Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload_eval_path Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload eval state path Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload client read Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 do read upload client Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 process request body Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 process request body Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 upload md5 variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload File size variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload Body Handler Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8

2019/11/13 14:21:52 [debug] 28086#28086: *3703 read upload clent request body Request address = 0000558ADDD4F780 Context = 0000000000000000
2019/11/13 14:21:52 [debug] 28086#28086: *3703 do read upload client Request address = 0000558ADDD4F780 Context = 0000000000000000


There appears to be an extra call to the request "read event" and although the request address has not changed the context address returned by:-

ngx_http_upload_ctx_t *u = ngx_http_get_module_ctx(r, ngx_http_upload_module);

Returns NULL, which causes any reference to the context table to cause a segment fault.

While it is possible to work round this by checking for a NULL context, the read event appears to be rouge when compared to the previous version of nginx, and I can only assume has been generated by code changes in 1.17.5.



Dave Brennan
Cyber Protection Senior Technologist

CORVID Protect Holdings Limited, trading as CORVID Protect, is registered in Guernsey, company number FC034204, whose registered office is at Royal Bank Place, 1 Glategny Esplanade, St Peter Port, Guernsey GY1 4ND. CORVID Protect Holdings Limited is a subsidiary company of Ultra Electronics Holdings plc registered in England and Wales, company number 02830397, whose registered office is at 35 Portman Square, London W1H 6LR.

Ultra Electronics is committed to safeguarding the privacy of all personal data: data privacy notice. Email communications may be monitored by us, as permitted by applicable law and regulations. This email is confidential and may also be privileged. If you have received this message in error you should notify the sender immediately by reply e-mail and delete the message from your system.
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel
Subject Author Views Posted

Seg fault in http read event handler caused by rouge event call without context

Dave Brennan 91 November 18, 2019 09:04AM

Re: Seg fault in http read event handler caused by rouge event call without context

Maxim Dounin 34 November 18, 2019 11:04AM

Re: Seg fault in http read event handler caused by rouge event call without context

Sergey Brester 34 November 18, 2019 11:50AM

Re: Seg fault in http read event handler caused by rouge event call without context

Maxim Dounin 36 November 18, 2019 12:12PM

Re: Seg fault in http read event handler caused by rouge event call without context

Sergey Brester 33 November 18, 2019 02:06PM

Re: Seg fault in http read event handler caused by rouge event call without context

Maxim Dounin 40 November 19, 2019 07:34AM

RE: Seg fault in http read event handler caused by rouge event call without context

Dave Brennan 59 November 19, 2019 09:12AM



Sorry, you do not have permission to post/reply in this forum.

Online Users

Guests: 82
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready