httpd memory leak

[tovmeod@gmail.com]

Hi,

I'm not sure if this is the place to ask for help with seastar, so point me in the right direction if this isn't the place.

I didn't touch c++ in more than 10 years so I realise my question could be on something obvious I'm failing to realize.

I copied the httpd example to my project, just running and stopping the server generates the memory leak

, it also prints warnings about "Exceptional future ignored".

I tried putting server->stop() inside try/catch, but it didn't help.

About the memory leak I tried deleting the server object before returning, but it made no difference.

Should I be catching the sigint somewhere?

below is my output, thanks.

Avraham

/home/avraham/fias/cmake-build-debug/fias -c2

WARNING: debug mode. Not for benchmarking or production

WARN  2017-02-27 04:54:39,535 seastar - Seastar compiled with default allocator, heap profiler not supported

WARN  2017-02-27 04:54:39,542 seastar - Seastar compiled with default allocator, heap profiler not supported

Seastar HTTP server listening on port 1800 ...

WARN  2017-02-27 04:55:33,336 [shard 0] seastar - Exceptional future ignored: std::system_error (error system:103, Software caused connection abort), backtrace: 0x555c7e330110, 0x555c7e33070e, 0x555c7e275b20, 0x555c7e261121, 0x555c7e2cd20b, 0x555c7e2cd4d9, 0x555c7e318904, 0x555c7e31f76e, 0x555c7e86f4fd, 0x555c7e225b05, 0x7ff912b0f3f0, 0x555c7e224239

WARN  2017-02-27 04:55:33,336 [shard 1] seastar - Exceptional future ignored: std::system_error (error system:103, Software caused connection abort), backtrace: 0x555c7e330110, 0x555c7e33070e, 0x555c7e275b20, 0x555c7e261121, 0x555c7e2cd20b, 0x555c7e2cd4d9, 0x555c7e318904, 0x555c7e31f76e, 0x555c7e32c564, 0x555c7e3f679b, 0x555c7e5b057f, 0x555c7e79a3de, 0x7ff9144676c9, 0x7ff912bf70ae

=================================================================

==29346==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 176 byte(s) in 2 object(s) allocated from:

    #0 0x7ff915868eb0 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc6eb0)

    #1 0x7ff9124c54a8 in lt__malloc (/usr/lib/x86_64-linux-gnu/libltdl.so.7+0x24a8)

Direct leak of 24 byte(s) in 1 object(s) allocated from:

    #0 0x7ff915869ef0 in operator new(unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc7ef0)

    #1 0x555c7e243f86 in httpd::http_server_control::http_server_control() (/home/avraham/fias/cmake-build-debug/fias+0x1994f86)

    #2 0x555c7e225192 in operator() /home/avraham/fias/fias/main.cpp:51

    #3 0x555c7e22b204 in _M_invoke /usr/include/c++/6/functional:1740

    #4 0x555c7e5b057f in std::function<void ()>::operator()() const /usr/include/c++/6/functional:2136

    #5 0x555c7e8988de in apply_helper<std::function<void ()>, std::tuple<>&&, std::integer_sequence<unsigned long> >::apply(std::function<void ()>&&, std::tuple<>&&) core/apply.hh:34

    #6 0x555c7e89896a in auto apply<std::function<void ()>>(std::function<void ()>&&, std::tuple<>&&) core/apply.hh:42

    #7 0x555c7e898a0e in std::enable_if<!is_future<std::result_of<std::function<void ()> ()>::type>::value, future<> >::type do_void_futurize_apply_tuple<std::function<void ()>>(std::function<void ()>&&, std::tuple<>&&) core/future.hh:1235

    #8 0x555c7e894dbf in future<> futurize<void>::apply<std::function<void ()>>(std::function<void ()>&&, std::tuple<>&&) core/future.hh:1255

    #9 0x555c7e8a1b5a in future<> future<>::then<std::function<void ()>, future<> >(std::function<void ()>&&)::{lambda(std::function<void ()>&&)#1}::operator()<future_state<> >(auto, std::function<void ()>&&) core/future.hh:872

    #10 0x555c7e8a1fd0 in continuation<future<> future<>::then<std::function<void ()>, future<> >(std::function<void ()>&&)::{lambda(std::function<void ()>&&)#1}>::run() core/future.hh:390

    #11 0x555c7e318904 in reactor::run_tasks(circular_buffer<std::unique_ptr<task, std::default_delete<task> >, std::allocator<std::unique_ptr<task, std::default_delete<task> > > >&) core/reactor.cc:2191

    #12 0x555c7e31f76e in reactor::run() core/reactor.cc:2629

    #13 0x555c7e86f4fd in app_template::run_deprecated(int, char**, std::function<void ()>&&) core/app-template.cc:131

    #14 0x555c7e225b05 in main /home/avraham/fias/fias/main.cpp:70

    #15 0x7ff912b0f3f0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x203f0)

SUMMARY: AddressSanitizer: 200 byte(s) leaked in 3 allocation(s).

Process finished with exit code 1

[Avi Kivity]

On 02/27/2017 05:30 AM, tov...@gmail.com wrote:

Hi,

I'm not sure if this is the place to ask for help with seastar, so point me in the right direction if this isn't the place.

Hi, this is indeed the right place.

I didn't touch c++ in more than 10 years so I realise my question could be on something obvious I'm failing to realize.

I copied the httpd example to my project, just running and stopping the server generates the memory leak

, it also prints warnings about "Exceptional future ignored".

I tried putting server->stop() inside try/catch, but it didn't help.

About the memory leak I tried deleting the server object before returning, but it made no difference.

Should I be catching the sigint somewhere?

below is my output, thanks.

Avraham

/home/avraham/fias/cmake-build-debug/fias -c2

WARNING: debug mode. Not for benchmarking or production

WARN  2017-02-27 04:54:39,535 seastar - Seastar compiled with default allocator, heap profiler not supported

WARN  2017-02-27 04:54:39,542 seastar - Seastar compiled with default allocator, heap profiler not supported

Seastar HTTP server listening on port 1800 ...

WARN  2017-02-27 04:55:33,336 [shard 0] seastar - Exceptional future ignored: std::system_error (error system:103, Software caused connection abort), backtrace: 0x555c7e330110, 0x555c7e33070e, 0x555c7e275b20, 0x555c7e261121, 0x555c7e2cd20b, 0x555c7e2cd4d9, 0x555c7e318904, 0x555c7e31f76e, 0x555c7e86f4fd, 0x555c7e225b05, 0x7ff912b0f3f0, 0x555c7e224239

WARN  2017-02-27 04:55:33,336 [shard 1] seastar - Exceptional future ignored: std::system_error (error system:103, Software caused connection abort), backtrace: 0x555c7e330110, 0x555c7e33070e, 0x555c7e275b20, 0x555c7e261121, 0x555c7e2cd20b, 0x555c7e2cd4d9, 0x555c7e318904, 0x555c7e31f76e, 0x555c7e32c564, 0x555c7e3f679b, 0x555c7e5b057f, 0x555c7e79a3de, 0x7ff9144676c9, 0x7ff912bf70ae

=================================================================

==29346==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 176 byte(s) in 2 object(s) allocated from:

    #0 0x7ff915868eb0 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc6eb0)

    #1 0x7ff9124c54a8 in lt__malloc (/usr/lib/x86_64-linux-gnu/libltdl.so.7+0x24a8)

Direct leak of 24 byte(s) in 1 object(s) allocated from:

    #0 0x7ff915869ef0 in operator new(unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc7ef0)

    #1 0x555c7e243f86 in httpd::http_server_control::http_server_control() (/home/avraham/fias/cmake-build-debug/fias+0x1994f86)

    #2 0x555c7e225192 in operator() /home/avraham/fias/fias/main.cpp:51

It's indeed a leak, here's the code:

        auto server = new http_server_control();

Amnon, I think this is yours.

    #3 0x555c7e22b204 in _M_invoke /usr/include/c++/6/functional:1740

    #4 0x555c7e5b057f in std::function<void ()>::operator()() const /usr/include/c++/6/functional:2136

    #5 0x555c7e8988de in apply_helper<std::function<void ()>, std::tuple<>&&, std::integer_sequence<unsigned long> >::apply(std::function<void ()>&&, std::tuple<>&&) core/apply.hh:34

    #6 0x555c7e89896a in auto apply<std::function<void ()>>(std::function<void ()>&&, std::tuple<>&&) core/apply.hh:42

    #7 0x555c7e898a0e in std::enable_if<!is_future<std::result_of<std::function<void ()> ()>::type>::value, future<> >::type do_void_futurize_apply_tuple<std::function<void ()>>(std::function<void ()>&&, std::tuple<>&&) core/future.hh:1235

    #8 0x555c7e894dbf in future<> futurize<void>::apply<std::function<void ()>>(std::function<void ()>&&, std::tuple<>&&) core/future.hh:1255

    #9 0x555c7e8a1b5a in future<> future<>::then<std::function<void ()>, future<> >(std::function<void ()>&&)::{lambda(std::function<void ()>&&)#1}::operator()<future_state<> >(auto, std::function<void ()>&&) core/future.hh:872

    #10 0x555c7e8a1fd0 in continuation<future<> future<>::then<std::function<void ()>, future<> >(std::function<void ()>&&)::{lambda(std::function<void ()>&&)#1}>::run() core/future.hh:390

    #11 0x555c7e318904 in reactor::run_tasks(circular_buffer<std::unique_ptr<task, std::default_delete<task> >, std::allocator<std::unique_ptr<task, std::default_delete<task> > > >&) core/reactor.cc:2191

    #12 0x555c7e31f76e in reactor::run() core/reactor.cc:2629

    #13 0x555c7e86f4fd in app_template::run_deprecated(int, char**, std::function<void ()>&&) core/app-template.cc:131

    #14 0x555c7e225b05 in main /home/avraham/fias/fias/main.cpp:70

    #15 0x7ff912b0f3f0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x203f0)

SUMMARY: AddressSanitizer: 200 byte(s) leaked in 3 allocation(s).

Process finished with exit code 1

--
You received this message because you are subscribed to the Google Groups "seastar-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to seastar-dev...@googlegroups.com.
To post to this group, send email to seast...@googlegroups.com.
Visit this group at https://groups.google.com/group/seastar-dev.
To view this discussion on the web visit https://groups.google.com/d/msgid/seastar-dev/bb282462-a7cc-4a7a-8b2b-03bb82564d39%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Amnon Heiman]

I'll take a look

To unsubscribe from this group and stop receiving emails from it, send an email to seastar-dev+unsubscribe@googlegroups.com.

[Tomasz Grabiec]

On Mon, Feb 27, 2017 at 4:30 AM, <tov...@gmail.com> wrote:

Hi,

I'm not sure if this is the place to ask for help with seastar, so point me in the right direction if this isn't the place.

I didn't touch c++ in more than 10 years so I realise my question could be on something obvious I'm failing to realize.

I copied the httpd example to my project, just running and stopping the server generates the memory leak

, it also prints warnings about "Exceptional future ignored".

As for the "Exceptional future ignored", can you try to resolve the backtrace attached to it?

See https://github.com/scylladb/scylla/wiki/How-to-resolve-backtrace

 

[Amnon Heiman]

On Mon, Feb 27, 2017 at 10:37 AM, Avi Kivity <a...@scylladb.com> wrote:

Yes, in main.cc

 http_server_control is created with new and never deleted.

 

    #3 0x555c7e22b204 in _M_invoke /usr/include/c++/6/functional:1740

    #4 0x555c7e5b057f in std::function<void ()>::operator()() const /usr/include/c++/6/functional:2136

    #5 0x555c7e8988de in apply_helper<std::function<void ()>, std::tuple<>&&, std::integer_sequence<unsigned long> >::apply(std::function<void ()>&&, std::tuple<>&&) core/apply.hh:34

    #6 0x555c7e89896a in auto apply<std::function<void ()>>(std::function<void ()>&&, std::tuple<>&&) core/apply.hh:42

    #7 0x555c7e898a0e in std::enable_if<!is_future<std::result_of<std::function<void ()> ()>::type>::value, future<> >::type do_void_futurize_apply_tuple<std::function<void ()>>(std::function<void ()>&&, std::tuple<>&&) core/future.hh:1235

    #8 0x555c7e894dbf in future<> futurize<void>::apply<std::function<void ()>>(std::function<void ()>&&, std::tuple<>&&) core/future.hh:1255

    #9 0x555c7e8a1b5a in future<> future<>::then<std::function<void ()>, future<> >(std::function<void ()>&&)::{lambda(std::function<void ()>&&)#1}::operator()<future_state<> >(auto, std::function<void ()>&&) core/future.hh:872

    #10 0x555c7e8a1fd0 in continuation<future<> future<>::then<std::function<void ()>, future<> >(std::function<void ()>&&)::{lambda(std::function<void ()>&&)#1}>::run() core/future.hh:390

    #11 0x555c7e318904 in reactor::run_tasks(circular_buffer<std::unique_ptr<task, std::default_delete<task> >, std::allocator<std::unique_ptr<task, std::default_delete<task> > > >&) core/reactor.cc:2191

    #12 0x555c7e31f76e in reactor::run() core/reactor.cc:2629

    #13 0x555c7e86f4fd in app_template::run_deprecated(int, char**, std::function<void ()>&&) core/app-template.cc:131

    #14 0x555c7e225b05 in main /home/avraham/fias/fias/main.cpp:70

    #15 0x7ff912b0f3f0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x203f0)

SUMMARY: AddressSanitizer: 200 byte(s) leaked in 3 allocation(s).

Process finished with exit code 1

--
You received this message because you are subscribed to the Google Groups "seastar-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to seastar-dev+unsubscribe@googlegroups.com.

[Avraham Serour]

thanks, it helped some, but I still have leaks:

btw, I just realized I've been replying to you directly and not to the list, sorry about that.

/home/avraham/fias/cmake-build-debug/fias -c2

WARNING: debug mode. Not for benchmarking or production

WARN  2017-03-05 12:05:47,371 seastar - Seastar compiled with default allocator, heap profiler not supported

WARN  2017-03-05 12:05:47,372 seastar - Seastar compiled with default allocator, heap profiler not supported

FIAS server listening on port 1800 ...

WARN  2017-03-05 12:05:50,488 [shard 0] seastar - Exceptional future ignored: std::system_error (error system:103, Software caused connection abort), backtrace: 0x5605f75fbaa2, 0x5605f75fc0a0, 0x5605f754cce6, 0x5605f753a501, 0x5605f7597aaf, 0x5605f7597d7d, 0x5605f75e4296, 0x5605f75eb100, 0x5605f7b3859b, 0x5605f74fcc9e, 0x7f1a9d38d3f0, 0x5605f74fbcd9

WARN  2017-03-05 12:05:50,488 [shard 1] seastar - Exceptional future ignored: std::system_error (error system:103, Software caused connection abort), backtrace: 0x5605f75fbaa2, 0x5605f75fc0a0, 0x5605f754cce6, 0x5605f753a501, 0x5605f7597aaf, 0x5605f7597d7d, 0x5605f75e4296, 0x5605f75eb100, 0x5605f75f7ef6, 0x5605f76c212d, 0x5605f787b7db, 0x5605f7a63336, 0x7f1a9ece56c9, 0x7f1a9d4750ae

=================================================================

==7114==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 176 byte(s) in 2 object(s) allocated from:

    #0 0x7f1aa00e6eb0 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc6eb0)

    #1 0x7f1a9cd434a8 in lt__malloc (/usr/lib/x86_64-linux-gnu/libltdl.so.7+0x24a8)

SUMMARY: AddressSanitizer: 176 byte(s) leaked in 2 allocation(s).

Process finished with exit code 1

On Sun, Mar 5, 2017 at 12:03 PM, Amnon Heiman <am...@scylladb.com> wrote:

I'm not sure why it didn't happen when we use it, but can you please try and modify httpd.hh and add

a destructor to http_server_control (line 415)

    ~http_server_control() {

        delete _server_dist;

    }

On Sun, Mar 5, 2017 at 11:54 AM, Avraham Serour <tov...@gmail.com> wrote:

Thanks, it no longer generates the runtime errors, but simply starting and stopping the server still leaks:

/home/avraham/fias/cmake-build-debug/fias -c2

WARNING: debug mode. Not for benchmarking or production

WARN  2017-03-05 11:52:54,035 seastar - Seastar compiled with default allocator, heap profiler not supported

WARN  2017-03-05 11:52:54,044 seastar - Seastar compiled with default allocator, heap profiler not supported

FIAS server listening on port 1800 ...

WARN  2017-03-05 11:52:57,139 [shard 0] seastar - Exceptional future ignored: std::system_error (error system:103, Software caused connection abort), backtrace: 0x563d9f71e88a, 0x563d9f71ee88, 0x563d9f66fbe0, 0x563d9f65d45f, 0x563d9f6ba897, 0x563d9f6bab65, 0x563d9f70707e, 0x563d9f70dee8, 0x563d9fc5b383, 0x563d9f61fc9e, 0x7fe29ffe23f0, 0x563d9f61ecd9

WARN  2017-03-05 11:52:57,140 [shard 1] seastar - Exceptional future ignored: std::system_error (error system:103, Software caused connection abort), backtrace: 0x563d9f71e88a, 0x563d9f71ee88, 0x563d9f66fbe0, 0x563d9f65d45f, 0x563d9f6ba897, 0x563d9f6bab65, 0x563d9f70707e, 0x563d9f70dee8, 0x563d9f71acde, 0x563d9f7e4f15, 0x563d9f99e5c3, 0x563d9fb8611e, 0x7fe2a193a6c9, 0x7fe2a00ca0ae

=================================================================

==6776==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 176 byte(s) in 2 object(s) allocated from:

    #0 0x7fe2a2d3beb0 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc6eb0)

    #1 0x7fe29f9984a8 in lt__malloc (/usr/lib/x86_64-linux-gnu/libltdl.so.7+0x24a8)

Direct leak of 24 byte(s) in 1 object(s) allocated from:

    #0 0x7fe2a2d3cef0 in operator new(unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc7ef0)

    #1 0x563d9f63afa0 in fias::server_control::server_control() (/home/avraham/fias/cmake-build-debug/fias+0x1913fa0)

    #2 0x563d9f61fc49 in main /home/avraham/fias/fias/main.cpp:50

    #3 0x7fe29ffe23f0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x203f0)

SUMMARY: AddressSanitizer: 200 byte(s) leaked in 3 allocation(s).

Process finished with exit code 1

On Sun, Mar 5, 2017 at 11:51 AM, Amnon Heiman <am...@scylladb.com> wrote:

put the server_control server outside of the call to app.run_deprecated

On Sun, Mar 5, 2017 at 11:50 AM, Avraham Serour <tov...@gmail.com> wrote:

int main(int ac, char** av) {

    app_template app;

    app.add_options()("port", bpo::value<uint16_t>()->default_value(1800),

                      "HTTP server port");

    return app.run_deprecated(ac, av, [&] {

        auto&& config = app.configuration();

        uint16_t port = config["port"].as<uint16_t>();

        server_control server;

//        auto rb = make_shared<api_registry_builder>("apps/httpd/");

        server.start().then([&server, port] {

            return server.listen(port);

        }).then([&server, port] {

            std::cout << "FIAS server listening on port " << port << " ...\n";

            engine().at_exit([&server] {

                return server.stop();

            });

        });

    });

}

On Sun, Mar 5, 2017 at 11:48 AM, Amnon Heiman <am...@scylladb.com> wrote:

can you show me the code?

On Sun, Mar 5, 2017 at 11:26 AM, Avraham Serour <tov...@gmail.com> wrote:

> Make sure you hold its reference in the lambda closure.

What do you mean? to pass the server object as reference?

In any case I did that, it causes runtime errors, it seems the framework expect to work with pointers, see below the full output:

/home/avraham/fias/cmake-build-debug/fias -c2

WARNING: debug mode. Not for benchmarking or production

WARN  2017-03-05 11:25:46,525 seastar - Seastar compiled with default allocator, heap profiler not supported

WARN  2017-03-05 11:25:46,533 seastar - Seastar compiled with default allocator, heap profiler not supported

net/api.hh:250:7: runtime error: reference binding to misaligned address 0x28250c334864 for type 'struct <unknown>', which requires 8 byte alignment

0x28250c334864: note: pointer points here

<memory cannot be printed>

net/api.hh:250:7: runtime error: constructor call on misaligned address 0x28250c334864 for type 'struct unique_ptr', which requires 8 byte alignment

0x28250c334864: note: pointer points here

<memory cannot be printed>

/usr/include/c++/6/bits/unique_ptr.h:206:7: runtime error: reference binding to misaligned address 0x28250c334864 for type 'struct <unknown>', which requires 8 byte alignment

0x28250c334864: note: pointer points here

<memory cannot be printed>

/usr/include/c++/6/bits/unique_ptr.h:207:74: runtime error: constructor call on misaligned address 0x28250c334864 for type 'struct __tuple_type', which requires 8 byte alignment

0x28250c334864: note: pointer points here

<memory cannot be printed>

/usr/include/c++/6/tuple:928:19: runtime error: reference binding to misaligned address 0x28250c334864 for type 'struct <unknown>', which requires 8 byte alignment

0x28250c334864: note: pointer points here

<memory cannot be printed>

/usr/include/c++/6/tuple:929:63: runtime error: constructor call on misaligned address 0x28250c334864 for type 'struct _Tuple_impl', which requires 8 byte alignment

0x28250c334864: note: pointer points here

<memory cannot be printed>

/usr/include/c++/6/tuple:211:19: runtime error: reference binding to misaligned address 0x28250c334864 for type 'struct <unknown>', which requires 8 byte alignment

0x28250c334864: note: pointer points here

<memory cannot be printed>

/usr/include/c++/6/tuple:213:38: runtime error: constructor call on misaligned address 0x28250c334864 for type 'struct _Head_base', which requires 8 byte alignment

0x28250c334864: note: pointer points here

<memory cannot be printed>

/usr/include/c++/6/tuple:114:19: runtime error: reference binding to misaligned address 0x28250c334864 for type 'struct <unknown>', which requires 8 byte alignment

0x28250c334864: note: pointer points here

<memory cannot be printed>

/usr/include/c++/6/tuple:115:42: runtime error: member access within misaligned address 0x28250c334864 for type 'struct _Head_base', which requires 8 byte alignment

0x28250c334864: note: pointer points here

<memory cannot be printed>

ASAN:DEADLYSIGNAL

=================================================================

==6299==ERROR: AddressSanitizer: SEGV on unknown address 0x28250c334864 (pc 0x55cd099c17a5 bp 0x7ffdddfa0910 sp 0x7ffdddfa08f0 T0)

    #0 0x55cd099c17a4 in std::_Head_base<0ul, net::server_socket_impl*, false>::_Head_base<net::server_socket_impl*>(net::server_socket_impl*&&) /usr/include/c++/6/tuple:115

    #1 0x55cd099c189e in std::_Tuple_impl<0ul, net::server_socket_impl*, std::default_delete<net::server_socket_impl> >::_Tuple_impl<net::server_socket_impl*, std::default_delete<net::server_socket_impl>, void>(net::server_socket_impl*&&, std::default_delete<net::server_socket_impl>&&) /usr/include/c++/6/tuple:213

    #2 0x55cd099c1976 in std::tuple<net::server_socket_impl*, std::default_delete<net::server_socket_impl> >::tuple<net::server_socket_impl*, std::default_delete<net::server_socket_impl>, true>(net::server_socket_impl*&&, std::default_delete<net::server_socket_impl>&&) /usr/include/c++/6/tuple:929

    #3 0x55cd099c1ab0 in std::unique_ptr<net::server_socket_impl, std::default_delete<net::server_socket_impl> >::unique_ptr(std::unique_ptr<net::server_socket_impl, std::default_delete<net::server_socket_impl> >&&) /usr/include/c++/6/bits/unique_ptr.h:207

    #4 0x55cd099be1aa in server_socket::server_socket(server_socket&&) net/api.hh:250

    #5 0x55cd092555e2 in void __gnu_cxx::new_allocator<server_socket>::construct<server_socket, server_socket>(server_socket*, server_socket&&) (/home/avraham/fias/cmake-build-debug/fias+0x19545e2)

    #6 0x55cd09241ac8 in void std::allocator_traits<std::allocator<server_socket> >::construct<server_socket, server_socket>(std::allocator<server_socket>&, server_socket*, server_socket&&) (/home/avraham/fias/cmake-build-debug/fias+0x1940ac8)

    #7 0x55cd0922e246 in void std::vector<server_socket, std::allocator<server_socket> >::emplace_back<server_socket>(server_socket&&) (/home/avraham/fias/cmake-build-debug/fias+0x192d246)

    #8 0x55cd09219af1 in std::vector<server_socket, std::allocator<server_socket> >::push_back(server_socket&&) (/home/avraham/fias/cmake-build-debug/fias+0x1918af1)

    #9 0x55cd0920f431 in fias::Server::listen(ipv4_addr) (/home/avraham/fias/cmake-build-debug/fias+0x190e431)

    #10 0x55cd09229a15 in future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}::operator()(unsigned int) const::{lambda()#1}::operator()() const (/home/avraham/fias/cmake-build-debug/fias+0x1928a15)

    #11 0x55cd09251743 in future<> futurize<future<> >::apply<future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}::operator()(unsigned int) const::{lambda()#1}&>(future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}::operator()(unsigned int) const::{lambda()#1}&) (/home/avraham/fias/cmake-build-debug/fias+0x1950743)

    #12 0x55cd0923e3c4 in futurize<std::result_of<future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}::operator()(unsigned int) const::{lambda()#1} ()>::type>::type smp::submit_to<future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}::operator()(unsigned int) const::{lambda()#1}>(unsigned int, futurize&&) (/home/avraham/fias/cmake-build-debug/fias+0x193d3c4)

    #13 0x55cd0922995f in future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}::operator()(unsigned int) const (/home/avraham/fias/cmake-build-debug/fias+0x192895f)

    #14 0x55cd09251df0 in future<> parallel_for_each<boost::range_detail::integer_iterator<unsigned int>, future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}>(boost::range_detail::integer_iterator<unsigned int>, future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}, future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}&&)::{lambda(parallel_for_each_state&)#1}::operator()(parallel_for_each_state) const (/home/avraham/fias/cmake-build-debug/fias+0x1950df0)

    #15 0x55cd0925239f in auto do_with<parallel_for_each_state, future<> parallel_for_each<boost::range_detail::integer_iterator<unsigned int>, future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}>(boost::range_detail::integer_iterator<unsigned int>, future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}, future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}&&)::{lambda(parallel_for_each_state&)#1}>(future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}&&, future<> parallel_for_each<boost::range_detail::integer_iterator<unsigned int>, future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}>(boost::range_detail::integer_iterator<unsigned int>, future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}, future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}&&)::{lambda(parallel_for_each_state&)#1}) (/home/avraham/fias/cmake-build-debug/fias+0x195139f)

    #16 0x55cd092525a9 in future<> parallel_for_each<boost::range_detail::integer_iterator<unsigned int>, future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}>(boost::range_detail::integer_iterator<unsigned int>, future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}, future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}&&) (/home/avraham/fias/cmake-build-debug/fias+0x19515a9)

    #17 0x55cd0923e60c in future<> parallel_for_each<boost::integer_range<unsigned int>, future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}>(boost::integer_range<unsigned int>&&, future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr)::{lambda(unsigned int)#1}&&) (/home/avraham/fias/cmake-build-debug/fias+0x193d60c)

    #18 0x55cd09229add in future<> seastar::sharded<fias::Server>::invoke_on_all<ipv4_addr>(future<> (fias::Server::*)(ipv4_addr), ipv4_addr) (/home/avraham/fias/cmake-build-debug/fias+0x1928add)

    #19 0x55cd0921505e in fias::server_control::listen(ipv4_addr) (/home/avraham/fias/cmake-build-debug/fias+0x191405e)

    #20 0x55cd091f95f9 in operator() /home/avraham/fias/fias/main.cpp:57

    #21 0x55cd091fc912 in apply /home/avraham/fias/cmake-build-debug/seastar-src/core/apply.hh:34

    #22 0x55cd091fc980 in apply<main(int, char**)::<lambda()>::<lambda()> > /home/avraham/fias/cmake-build-debug/seastar-src/core/apply.hh:42

    #23 0x55cd091fc9f3 in apply<main(int, char**)::<lambda()>::<lambda()> > /home/avraham/fias/cmake-build-debug/seastar-src/core/future.hh:1267

    #24 0x55cd09207422 in operator()<future_state<> > /home/avraham/fias/cmake-build-debug/seastar-src/core/future.hh:872

    #25 0x55cd09207551 in run /home/avraham/fias/cmake-build-debug/seastar-src/core/future.hh:390

    #26 0x55cd092e1030 in reactor::run_tasks(circular_buffer<std::unique_ptr<task, std::default_delete<task> >, std::allocator<std::unique_ptr<task, std::default_delete<task> > > >&) core/reactor.cc:2191

    #27 0x55cd092e7e9a in reactor::run() core/reactor.cc:2629

    #28 0x55cd09835335 in app_template::run_deprecated(int, char**, std::function<void ()>&&) core/app-template.cc:131

    #29 0x55cd091f9c78 in main /home/avraham/fias/fias/main.cpp:64

    #30 0x7f5ca1d0b3f0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x203f0)

    #31 0x55cd091f8cd9 in _start (/home/avraham/fias/cmake-build-debug/fias+0x18f7cd9)

AddressSanitizer can not provide additional info.

SUMMARY: AddressSanitizer: SEGV /usr/include/c++/6/tuple:115 in std::_Head_base<0ul, net::server_socket_impl*, false>::_Head_base<net::server_socket_impl*>(net::server_socket_impl*&&)

==6299==ABORTING

Process finished with exit code 1

On Sun, Mar 5, 2017 at 8:59 AM, Amnon Heiman <am...@scylladb.com> wrote:

On Sun, Mar 5, 2017 at 8:43 AM, Avraham Serour <tov...@gmail.com> wrote:

I am deleting on at_exit, but it is still leaking, where should I delete?

It seems right, but instead, just make it a stack object outside of app.run_deprecated (drop the pointer alltogether)

Make sure you hold its reference in the lambda closure.

 

.then([server, port] {

            std::cout << "server listening on port " << port << " ...\n";

            engine().at_exit([server] {

                auto rc = server->stop();

                delete server;

                return rc;

            });

        });

[Avraham Serour]

I moved back to using a pointer but declared the variable outside run_deprecated and also added the destructor to server_control (see below), it doesn't complain about any leaks anymore, but it does still prints the backtrace which I am still not able to resolve, addr2line just prints "?? ??:0"

int main(int ac, char** av) {

    app_template app;

    app.add_options()("port", bpo::value<uint16_t>()->default_value(1800),

                      "HTTP server port");

    auto server = new fias::server_control();

    return app.run_deprecated(ac, av, [&] {

        auto&& config = app.configuration();

        uint16_t port = config["port"].as<uint16_t>();

//        auto rb = make_shared<api_registry_builder>("apps/httpd/");

        server->start().then([server, port] {

            return server->listen(port);

        }).then([server, port] {

            std::cout << "FIAS server listening on port " << port << " ...\n";

            engine().at_exit([server] {

                auto rc = server->stop();

                delete server;

                return rc;

            });

        });

    });

}

/home/avraham/fias/cmake-build-debug/fias -c2

WARNING: debug mode. Not for benchmarking or production

WARN  2017-03-05 12:51:40,795 seastar - Seastar compiled with default allocator, heap profiler not supported

WARN  2017-03-05 12:51:40,811 seastar - Seastar compiled with default allocator, heap profiler not supported

FIAS server listening on port 1800 ...

fias: /home/avraham/fias/cmake-build-debug/seastar-src/core/sharded.hh:327: seastar::sharded<Service>::~sharded() [with Service = fias::Server]: Assertion `_instances.empty()' failed.

Aborting on shard 0.

Backtrace:

  0x0000555e2626631f

  0x0000555e25faeebf

  0x0000555e25faefc2

  0x0000555e25ffcca9

  0x0000555e260808c7

  0x0000555e26080962

  0x00007fe0446b762f

  0x00007fe042d6a7ee

  0x00007fe042d6c3e9

  0x00007fe042d62bb6

  0x00007fe042d62c61

  0x0000555e25f31d3e

  0x0000555e25f1e02e

  0x0000555e25f0265d

  0x0000555e25f05d23

  0x0000555e2628af64

  0x0000555e2605f21e

  0x0000555e2605f556

  0x0000555e25fe2958

  0x0000555e25fe4dda

  0x0000555e25fecf85

  0x0000555e260c2d53

  0x0000555e2628180b

  0x0000555e25fae6a3

  0x0000555e2608f3bf

  0x0000555e2628180b

  0x0000555e25faec8f

  0x0000555e2626cd89

  0x0000555e25ff27bb

  0x0000555e25feeef7

  0x0000555e260c541f

  0x0000555e262a266b

  0x0000555e25ff1407

  0x0000555e2653e5cb

  0x0000555e25f02cdc

  0x00007fe042d553f0

  0x0000555e25f01cd9

Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

[Tomasz Grabiec]

On Sun, Mar 5, 2017 at 12:00 PM, Avraham Serour <tov...@gmail.com> wrote:

I moved back to using a pointer but declared the variable outside run_deprecated and also added the destructor to server_control (see below), it doesn't complain about any leaks anymore, but it does still prints the backtrace which I am still not able to resolve, addr2line just prints "?? ??:0"

Try running under GDB and run "bt" when it aborts.

Maybe addr2line fails because your main object is not mapped at the standard address (addresses have this 0x0000555e2 prefix). If you knew the mapping, you could adjust addr2line.

That suggests that some sharded<> object goes out of scope without being stop()'ed

Maybe exception is thrown before stop() is reached. Try "catch throw" in GDB.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/seastar-dev/CAFWa6t%2B42U3qWknHZJyonhf%2BFWw5W0eQ-UAh8OQndcFeVJWcAw%40mail.gmail.com.