[erlang-questions] Question about http client
mtalyans
I am a rank beginner, so please bear with me:
I am fetching Web content from a set of URLs using http:request. For most of URLs everything works fine; however for some of them I see error reports on the console that look like so:
<0.49.0> spawned pid <0.50.0> (here my main process spawns another one to do the fetch)
=ERROR REPORT==== 15-Apr-2009::13:29:50 ===
** Generic server <0.51.0> terminating
** Last message in was {tcp_closed,#Port<0.1570>}
** When Server state == {state,
{request,#Ref<0.0.0.67>,<0.50.0>,0,http,
{"message.real.com",80},
"/Msg/7.0/win32",[],get,
{http_request_h,undefined,"keep-alive",
undefined,undefined,undefined,undefined,
undefined,undefined,undefined,undefined,
undefined,undefined,undefined,undefined,
undefined,undefined,"message.real.com",
undefined,undefined,undefined,undefined,
undefined,undefined,undefined,undefined,
undefined,[],undefined,undefined,
undefined,undefined,"0",undefined,
undefined,undefined,undefined,undefined,
undefined,[]},
{[],[]},
{http_options,"HTTP/1.1",5000,true,[],
undefined,true},
"http://message.real.com/Msg/7.0/win32",[],
none,[]},
{tcp_session,
{{"message.real.com",80},<0.51.0>},
false,http,#Port<0.1570>,1,keep_alive},
undefined,undefined,undefined,
{httpc_response,parse,[nolimit,true]},
{[],[]},
{[],[]},
new,[],nolimit,nolimit,
{options,
{undefined,[]},
0,2,5,120000,2,disabled,enabled,false},
{timers,
[{#Ref<0.0.0.67>,#Ref<0.0.0.77>}],
undefined},
httpc_manager,undefined}
** Reason for termination ==
** session_remotly_closed
<0.50.0>: normal clause "ERLANG: Request error session_remotly_closed on http://message.real.com/Msg/7.0/win32" (here my fetched process gets results)
As far as I can tell, session has been remotely closed. I don’t mind a failure like this, but I would like to get rid of the error reports, as I am fetching many, many URLs. I set verbose option to false, and I also catch everything around the call to http:request (in fact, the last line in the snippet is my own printout of the normal clause in the try/catch), all to no avail. It seems that what is terminating is not the process I have spawned to do the fetch.
In other cases, I get content properly fetched, and then get an error report with a timeout:
<0.49.0> spawned pid <0.50.0>
<0.50.0>: normal clause "Normal completion"
=ERROR REPORT==== 15-Apr-2009::13:35:05 ===
** Generic server <0.51.0> terminating
** Last message in was {timeout,#Ref<0.0.0.67>}
** When Server state == {state,undefined,
{tcp_session,
{{"links.preplogic.com",80},<0.51.0>},
false,http,#Port<0.1570>,1,keep_alive},
undefined,undefined,undefined,undefined,
{[],[]},
{[],[]},
keep_alive,[],nolimit,nolimit,
{options,
{undefined,[]},
0,2,5,120000,2,disabled,enabled,false},
{timers,[],#Ref<0.0.0.78>},
httpc_manager,undefined}
** Reason for termination ==
** {{badrecord,request},
[{httpc_handler,handle_info,2},
{gen_server,handle_msg,5},
{proc_lib,init_p_do_apply,3}]}
Could someone in the know help me to solve this issue?
Thanks in advance!
Bengt Kleberg
It seems as if you are getting error reports from the SASL application
(http://www.erlang.org/doc/man/sasl_app.html). If this is the case you
could either turn it off:
"The following configuration parameters are defined for the SASL
application. See app(4) for more information about configuration
parameters:
sasl_error_logger = Value <optional>
Value is one of:
tty
Installs sasl_report_tty_h in the error logger. This is the default
option.
{file,FileName}
Installs sasl_report_file_h in the error logger. This makes all reports
go to the file FileName. FileName is a string.
false
No SASL error logger handler is installed."
or you could replace it with one that is less verbose:
"The default simple handler is called alarm_handler and it may be
exchanged by calling gen_event:swap_handler/3 as
gen_event:swap_handler(alarm_handler, {alarm_handler, swap},
{NewHandler, Args}). NewHandler:init({Args, {alarm_handler, Alarms}}) is
called. Refer to gen_event(3) for further details."
bengt
> _______________________________________________
> erlang-questions mailing list
> erlang-q...@erlang.org
> http://www.erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://www.erlang.org/mailman/listinfo/erlang-questions
mats cronqvist
> Hi All,
>
> I am a rank beginner, so please bear with me:
>
> I am fetching Web content from a set of URLs using http:request. For
> most of URLs everything works fine; however for some of them I see
> error reports on the console that look like so:
... tons of crap...
> ** Reason for termination ==
> ** session_remotly_closed
>
> As far as I can tell, session has been remotely closed.
that is indeed what's happening.
> I don’t mind a failure like this, but I would like to get rid of
> the error reports, as I am fetching many, many URLs.
it is a bizarre design decision to print an error report for a
perfectly normal event. there has been some complaining about this
on erlang-bugs (by me among others) to no avail.
The only sane solution (turning off sasl is insane) I've been able
to come up with is to patch; IIRC you have to replace the atom
'session_remotly_closed' with 'shutdown' in a few lines like this;
httpc_handler.erl: {stop, session_remotly_closed, State};
this looks like a "real" bug. handle_info/2 expects a to be handed a
record, but it gets an atom, namely 'request'. This will happens if
you type 'request' instead of 'Request' at the right place...
I'll CC this to erlang-bugs, in case the OTP guys miss it.
mats