[erlang-questions] Possible leak in inets httpc.

[Andrey Popp]

Hello,

I have application that makes huge number of HTTP requests with httpc
and I have spotted that `httpc_manager__handler_db` ETS table growing
in size and doesn't clean up. That is the example records from this
table (ets:i(httpc_manager__handler_db)):

<1 > {handler_info,#Ref<0.0.0.18800>,undefined,<0.177.0>,undefined,operational}
<2 > {handler_info,#Ref<0.0.0.35242>,undefined,<0.3075.0>,undefined,operational}
<3 > {handler_info,#Ref<0.0.0.61713>,undefined,<0.5755.0>,undefined,operational}
<4 > {handler_info,#Ref<0.0.0.68114>,undefined,<0.6943.0>,undefined,operational}
<5 > {handler_info,#Ref<0.0.0.85303>,undefined,<0.8305.0>,undefined,operational}

and processes with handler pids are no longer alive. Also, note that I
have no timeout setting in my http:request HTTPOptions, but I think
inets should not leak even in that case.

Thanks.

--
Andrey Popp

phone: +7 911 740 24 91
e-mail: 8ma...@gmail.com

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questio...@erlang.org

[Andrey Popp]

It seems it has been fixed with [1] in inets-5.3.2. Sorry.

[1]: http://github.com/erlang/otp/commit/91c89d54d45989a85367f10d5902b9b508754a49

[Anthony Molinaro]

Hi, anyone know if its safe to backport this fix to R13B04? I'm in the process
of deploying R13B04 as an upgrade from R12B05 and since I make heavy use of
httpc this will bite me. Ideally, I can just grab the appropriate files from
git and plop them into my R13B04 when I build an rpm (of course I need to
remember enough git to do that :) ), so wondering if it's possible or if
there are deps outside of the inets directory which matter?

Thanks,

-Anthony

--
------------------------------------------------------------------------
Anthony Molinaro <anth...@alumni.caltech.edu>

[Anthony Molinaro]

Well, looks like the answer is yes and no. I was able to get a patch
which applied cleanly, however, the problem does not appear to be fixed.
The ets table still fills up with every request that times out
(I unfortunately don't have a standalone test, but I've tested
repeatedly and it always seem to get an extra entry for every request
that times out).

This is sort of a show stopper for me upgrading, so I'll be trying to
figure out a fix, but if anyone more familiar with the code wants to
help me out, it would be appreciated.

Thanks,

-Anthony

[Andrey Popp]

We have updated to R14 (thanks to deb packages in debian squeeze
repository) and all seems to be ok for now.

[Micael Karlberg]

This problem was fixed in inets-5.3.2. The fix was reported by Lev Walkin:

http://www.erlang.org/cgi-bin/ezmlm-cgi/2/1766

Unfortunately, github was not updated in one commit, but in several, here
are some of them:

http://github.com/erlang/otp/commit/4b87b22ce97abf2759eb551222a862e17c5f4dcb
http://github.com/erlang/otp/commit/91c89d54d45989a85367f10d5902b9b508754a49

So you have to do a bit of cut-and-paste. The last version of inets in
R13B was 5.3.3,
but I don't know if that made it into github.

Regards,
/BMK

[Anthony Molinaro]

Yeah, I managed to get the entire set of patches, basically used

git diff 201ccdd08bd2a591fe1e348a9a1df3d94a3606e4 141ff3c087ca856d21d0 lib/inets

Which seemed to get them all. However, I still see memory leaked

Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0]
[kernel-poll:false]

Eshell V5.7.5 (abort with ^G)
1> inets:start().
ok
2> ets:i().
...
16402 httpc_manager__session_cookie_db bag 0 316 httpc_manager
httpc_manager__handler_db httpc_manager__handler_db set 0 316 httpc_manager
httpc_manager__session_db httpc_manager__session_db set 0 316 httpc_manager
...
3> http:request (post, { "http://www.example.com", [], "application/octet-stream", <<"hello">>}, [{connect_timeout, 100},{timeout, 10}], [{body_format, binary} ]).
{error,timeout}
4> ets:i().
...
16402 httpc_manager__session_cookie_db bag 0 316 httpc_manager
httpc_manager__handler_db httpc_manager__handler_db set 1 333 httpc_manager
httpc_manager__session_db httpc_manager__session_db set 0 316 httpc_manager
...
5> ets:i(httpc_manager__handler_db).
<1 > {handler_info,#Ref<0.0.0.142>,undefined,<0.60.0>,<0.33.0>,operational}

6> m(inets).
Module inets compiled: Date: July 8 2010, Time: 06.45
Compiler options: [{d,'SERVER_SOFTWARE',"inets/5.3.3"},
{cwd,"/home/molinaro/projects/work/vendor/erlang/R13B04/rpmbuild/BUILD/otp_src_R13B04/lib/inets/src/inets_app"},
{outdir,"/home/molinaro/projects/work/vendor/erlang/R13B04/rpmbuild/BUILD/otp_src_R13B04/lib/inets/src/inets_app/../../ebin"},
{attribute,insert,app_vsn,"inets-5.3.3"},
{parse_transform,sys_pre_attributes},
debug_info]
Object file: /usr/lib64/erlang/lib/inets-5.3.3/ebin/inets.beam
...

So it seems if you connect but then timeout you leak some memory even with
5.3.3. I've not tried 5.4 which is in dev but the only changes there seem
to be related to ssl...

Can anyone else confirm this is still broken?

-Anthony

[Anthony Molinaro]

Well, the leak was there in a big way under production traffic patterns, so
I switched from httpc to ibrowse. All the leaks went away. Just an FYI
for anyone out there making a decision between httpc and ibrowse.

-Anthony