Hi Hugo,

This is great to see your interest to nginx tests.

On 14.01.2022 18:45, Hugo Lefeuvre wrote:
> Hello!
>
> As part of a research project, I measured the coverage achieved by Nginx's
> test-suite using gcov. Taking a look at the results, my colleagues and
> myself were somewhat surprised to realize that the coverage capped at about
> 70% line coverage and 81% function coverage; we expected something closer
> to 90% line coverage. core/nginx.c for example only gets 268 lines / 584

This is more or less in line with our numbers. Just from today report
for mainline: 77.8% for lines coverage and 93.1% for functions. These
figures include njs code though.

> covered. Similarly, more than half of core/ngx_resover.c (1042 lines /
> 2138) is not covered.
> > I was wondering if I did something wrong in my measurements, if this is a
> known weakness of the test-suite, and in the latter case, if this is
> something that the Nginx project is open to receiving contributions on.
>
Probably not a weakness but unjustified expectations? :-)

I think the biggest non-covered part comes from various errors paths.
Some of them could be hard to trigger. For example, for memory
allocation failures we use separate nodes with modified ngx_palloc.c
which enables random memory allocation errors. This is not a part of
the standard test suite though.

Anyway, any meaningful contributions in this area will be highly
appreciated.

http://nginx.org/en/docs/contributing_changes.html

There are several things to keep in mind: we want to keep the suite
compact as it is now and be able to have it integrated into CI/CD
systems easily.

Thanks,

Maxim

--
Maxim Konovalov
_______________________________________________
nginx-devel mailing list -- nginx-devel@nginx.org
To unsubscribe send an email to nginx-devel-leave@nginx.org