Welcome! Log In Create A New Profile


Re: On test-suite coverage

Hugo Lefeuvre
January 15, 2022 04:54AM
Hi Maxim,

On Fri, Jan 14, 2022 at 07:33:20PM +0300, Maxim Konovalov wrote:
> This is more or less in line with our numbers. Just from today report for
> mainline: 77.8% for lines coverage and 93.1% for functions. These figures
> include njs code though.
> > I was wondering if I did something wrong in my measurements, if this is a
> > known weakness of the test-suite, and in the latter case, if this is
> > something that the Nginx project is open to receiving contributions on.
> Probably not a weakness but unjustified expectations? :-)
> I think the biggest non-covered part comes from various errors paths.
> Some of them could be hard to trigger. For example, for memory
> allocation failures we use separate nodes with modified ngx_palloc.c
> which enables random memory allocation errors. This is not a part of
> the standard test suite though.

Thanks for confirming the numbers! Makes sense. We made the same
observation with error paths, and it's nice to confirm it. I suppose that
stimulating some of these error paths would require approaches (syscall
interception, some degree of mocking) that might be in contradiction with
the requirements you listed below (compactness, CI/CD).

> Anyway, any meaningful contributions in this area will be highly
> appreciated.
> http://nginx.org/en/docs/contributing_changes.html
> There are several things to keep in mind: we want to keep the suite compact
> as it is now and be able to have it integrated into CI/CD systems easily.

Alright, we will definitely attempt to upstream any test that we might have
to write for this project. :-)


Hugo Lefeuvre (hle) | www.owl.eu.com
RSA4096_ 360B 03B3 BF27 4F4D 7A3F D5E8 14AA 1EB8 A247 3DFD
ed25519_ 37B2 6D38 0B25 B8A2 6B9F 3A65 A36F 5357 5F2D DC4C
nginx-devel mailing list -- nginx-devel@nginx.org
To unsubscribe send an email to nginx-devel-leave@nginx.org
Subject Author Views Posted

On test-suite coverage

Hugo Lefeuvre 193 January 14, 2022 10:48AM

Re: On test-suite coverage

Maxim Konovalov 75 January 14, 2022 11:34AM

Re: On test-suite coverage

Hugo Lefeuvre 88 January 15, 2022 04:54AM

Sorry, you do not have permission to post/reply in this forum.

Online Users

Guests: 55
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready