segfault in test_deflate_buckets

[Andreas Stieger]

Hello,

I am getting a crashing test (SIGSEGV) in 1.3.8 (not 1.3.7) and trunk. I
have narrowed this down to r2419 from trunk on test/test_buckets.c,
the change to test_deflate_buckets specifically.

> Program received signal SIGSEGV, Segmentation fault.
> serf_bucket_deflate_create (stream=stream@entry=0x100a0, allocator=allocator@entry=0xb96f26b8, format=format@entry=0) at buckets/deflate_buckets.c:90
> 90 ctx->stream_status = APR_SUCCESS;
> (gdb) backtrace
> #0 serf_bucket_deflate_create (stream=stream@entry=0x100a0, allocator=allocator@entry=0xb96f26b8, format=format@entry=0) at buckets/deflate_buckets.c:90
> #1 0x0805379a in deflate_buckets (nr_of_loops=242, tc=0x80650d0) at test/test_buckets.c:1327
> #2 test_deflate_buckets (tc=0x80650d0) at test/test_buckets.c:1382
> #3 0x0804b9b4 in CuTestRun (tc=tc@entry=0x80650d0) at test/CuTest.c:173
> #4 0x0804c054 in CuSuiteRun (testSuite=testSuite@entry=0x8062190) at test/CuTest.c:357
> #5 0x0804b2fb in main (argc=1, argv=0xbffff164) at test/test_all.c:95

and...

> Program received signal SIGSEGV, Segmentation fault.
> create_aggregate (allocator=allocator@entry=0xb96f26b8) at buckets/aggregate_buckets.c:73
> 73 ctx->list = NULL;
> (gdb) backtrace
> #0 create_aggregate (allocator=allocator@entry=0xb96f26b8) at buckets/aggregate_buckets.c:73
> #1 0xb7fc85eb in serf_bucket_aggregate_create (allocator=allocator@entry=0xb96f26b8) at buckets/aggregate_buckets.c:88
> #2 0x08053782 in deflate_buckets (nr_of_loops=242, tc=0x80650d0) at test/test_buckets.c:1326
> #3 test_deflate_buckets (tc=0x80650d0) at test/test_buckets.c:1382
> #4 0x0804b9b4 in CuTestRun (tc=tc@entry=0x80650d0) at test/CuTest.c:173
> #5 0x0804c054 in CuSuiteRun (testSuite=testSuite@entry=0x8062190) at test/CuTest.c:357
> #6 0x0804b2fb in main (argc=1, argv=0xbffff164) at test/test_all.c:95

Both referenced points in code accessing an object allocated with
serf_bucket_mem_alloc. strace ends with:

> mmap2(NULL, 102400, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb96b9000
> mmap2(NULL, 102400, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb96d2000
> mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x12000
> mmap2(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 EPERM (Operation not permitted)
> --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x200c} ---
> +++ killed by SIGSEGV +++

Options used:
> scons 'CFLAGS=-fomit-frame-pointer -fmessage-length=0 -grecord-gcc-switches -fstack-protector -O2 -Wall -D_FORTIFY_SOURCE=2 -funwind-tables -fasynchronous-unwind-tables -g' check -j2

openSUSE 13.1 i586 gcc 4.8.1. There are more SIGSEGV in trunk, maybe
similar, but this being in the release worries me just now. I'll look at
this further soon but maybe someone familiar with the code can make
sense of the above faster.

Andreas

[Lieven Govaerts]

Can this be an out of memory error? The test loops 1 million times
over compress/decompress steps, and does all allocations in the same
pool.

Nothing has changed in the create buckets code paths.

Lieven

[Ivan Zhakov]

Hi Lieven,

Serf trunk test failing with out of memory on Windows 32-bit build:
[[[
.....................F

There was 1 failure:
1) test_deflate_buckets:
c:\ivan\svn\serf-trunk\test\test_buckets.c:1454: expected <0> but was
<2001
4>

!!!FAILURES!!!
Runs: 22 Passes: 21 Fails: 1
]]]


Serf 1.3.8 also fails in Windows 64-bit build:
[[[
D:\builder\server\Externals\svn-httpd\x64\serf\Release>test_all.exe buckets
...................F

There was 1 failure:
1) test_deflate_buckets: test\test_buckets.c:1363: expected <0> but was <20014>

!!!FAILURES!!!
Runs: 20 Passes: 19 Fails: 1
]]]

--
Ivan Zhakov

[Andreas Stieger]

The messages previously quoted are failed memory allocations, but it
looks like the tests are leaking memory through SSL_new:
test_https_server_setup -> init_ssl -> SSL_new
ssl_reset -> init_ssl -> SSL_new
with an apparent lack of calls to SSL_free in the test code. It looks
like this was the case previously and was triggered by the increased
iterations?

57,324 (468 direct, 56,856 indirect) bytes in 1 blocks are definitely
lost in loss record 1,127 of 1,129
at 0x4029EAD: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
by 0x411C24B: ??? (in /lib/libcrypto.so.1.0.0)
by 0x411C861: CRYPTO_malloc (in /lib/libcrypto.so.1.0.0)
by 0x40B480A: SSL_new (in /lib/libssl.so.1.0.0)
by 0x8059250: init_ssl.isra.1 (test_sslserver.c:154)
by 0x804C936: test_https_server_setup (test_util.c:216)
{many callers}

Andreas

[Ivan Zhakov]

On 22 October 2014 01:22, Andreas Stieger <andreas...@gmx.de> wrote:
> On 21/10/14 11:17, Lieven Govaerts wrote:
>> On Tue, Oct 21, 2014 at 2:56 AM, Andreas Stieger <andreas...@gmx.de> wrote:
>>> Hello,
>>>
>>> I am getting a crashing test (SIGSEGV) in 1.3.8 (not 1.3.7) and trunk. I
>>> have narrowed this down to r2419 from trunk on test/test_buckets.c,
>>> the change to test_deflate_buckets specifically.
>>>

Andreas,

I've added iterpool in this specific test in r2445: could you please
try trunk build and confirm that it doesn't fails now?


--
Ivan Zhakov

[Andreas Stieger]

Hello,

On 23/10/14 17:29, Ivan Zhakov wrote:
> On 22 October 2014 01:22, Andreas Stieger <andreas...@gmx.de> wrote:
>> On 21/10/14 11:17, Lieven Govaerts wrote:
>>> On Tue, Oct 21, 2014 at 2:56 AM, Andreas Stieger <andreas...@gmx.de> wrote:
>>>> Hello,
>>>>
>>>> I am getting a crashing test (SIGSEGV) in 1.3.8 (not 1.3.7) and trunk. I
>>>> have narrowed this down to r2419 from trunk on test/test_buckets.c,
>>>> the change to test_deflate_buckets specifically.
>>>>

> I've added iterpool in this specific test in r2445: could you please
> try trunk build and confirm that it doesn't fails now?

With r2445 merged to ^/branches/1.3.x the tests pass okay. In trunk
there is now different behavior (the other SIGSEGV mentioned) which I
will share separately.

I propose the comment style fix in r2443 for back-porting as it breaks
strict compilers in ^/branches/1.3.x

Thanks,
Andreas