How to ignore Content-Length: 0 in mojo:ua

[Ivan Korjavin]

 I have web service with strange server (Server: hpd/1.0.9.2), and it always return me  Content-Length: 0 on my request.

my $ua          = Mojo::UserAgent->new;
my $res  = $ua->get($uri)->res;

And $res always empty.

But wget ignore this content-length and show me tight document.

How to ignore this in Mojo:Ua ?

[Ivan Korjavin]

I add more debug with morbo and MOJO_USERAGENT_DEBUG=1

there is log:

====

-- Blocking request (http://myserver:11080/hydra?command=check&txn_id=9&account=210220&sum=10.00)
-- Connect (http:myserver:11080)
-- Client >>> Server (http://myserver:11080/hydra?command=check&txn_id=9&account=210220&sum=10.00)
GET /hydra?command=check&txn_id=9&account=210220&sum=10.00 HTTP/1.1
User-Agent: Mojolicious (Perl)
Connection: keep-alive
Accept-Encoding: gzip
Content-Length: 0
Host: myserver:11080


-- Client <<< Server (http://myserver:11080/hydra?command=check&txn_id=9&account=210220&sum=10.00)
HTTP/1.0 200 OK

-- Client <<< Server (http://myserver:11080/hydra?command=check&txn_id=9&account=210220&sum=10.00)
Server: hpd/1.0.9.2
Date: Tue, 03 Jun 2014 07:23:00 GMT

<?xml version="1.0" encoding="utf-8"?><response><txn_id>9</txn_id><result>5</result><comment>Неверные данные</comment></response>

===

But
$ua->get($uri)->res;

is

HTTP/1.0 200 OK
Date: Tue, 03 Jun 2014 07:20:09 GMT
Content-Length: 0
Server: hpd/1.0.9.2

without data.

[Roland Lammel]

The server is definitly omitting the Content-Length header here, which seems to case the UA to not consider the body. in HTTP/1.0 it seems there is no clear indication on how a response should be handled without Content-Length (on the request side HTTP/1.0 requires a content-length to be set, as Mojo::UA does).

RFC: http://www.w3.org/Protocols/HTTP/1.0/spec.html

I would definitly try to correctly send a content-length for your hpd server, to ensure it is parsed correctly on all clients.

Otherwise you verify the spec and if you find the Mojo::UA behaviour not according to the spec, provide a spec test.

I personally never had any issues with paring the response (but also never had a server not supplying the content-length)

+rl
--
Roland Lammel
QuikIT - IT Lösungen - flexibel und schnell
Web: http://www.quikit.at
Phone: +43 (676) 9737845
Email: r...@quikit.at

"Enjoy your job, make lots of money, work within the law. Choose any two."

--
You received this message because you are subscribed to the Google Groups "Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mojolicious...@googlegroups.com.
To post to this group, send email to mojol...@googlegroups.com.
Visit this group at http://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.

[Ivan Korjavin]

Thank you Roland.

But I can't do anything with this server, its not mine, its third party integration.

May be, I can switch mojo:ua for something less strict?

I need advice.

вторник, 3 июня 2014 г., 14:01:27 UTC+6 пользователь Roland Lammel написал:

[Roland Lammel]

To bad, in this case you probably have to dig into Mojo::Transaction, Mojo::Transaction::HTTP, Mojo::Message and Mojo::Message::Response to check, why the content is not read/parsed (according to the spec for the server, the content should be read until the disconnect is received, so there is a chance that it is read, but I haven't checked to sources).

Hope this helps a little

+rl
--
Roland Lammel
QuikIT - IT Lösungen - flexibel und schnell
Web: http://www.quikit.at
Phone: +43 (676) 9737845
Email: r...@quikit.at

"Enjoy your job, make lots of money, work within the law. Choose any two."

[Tekki]

Have you tried this? http://mojolicio.us/perldoc/Mojo/Content#relaxed

[Ivan Korjavin]

Thank you Tekki, but I didn't understand where is Content here.

Please look at this script:

use Mojo::Message::Response;

# Parse bad Content-Length
my $res = Mojo::Message::Response->new;
$res->parse("HTTP/1.0 200 OK\x0d\x0a");
$res->parse("Content-Length: 0\x0d\x0a");
$res->parse("Content-Type: text/plain\x0d\x0a\x0d\x0a");
$res->parse('Hello World!');
print $res->to_string;

It return:

HTTP/1.0 200 OK
Date: Tue, 03 Jun 2014 12:53:19 GMT
Content-Length: 0
Content-Type: text/plain

Body is lost due to Content-length.

I searching the way for ignore that header.

2014-06-03 18:35 GMT+06:00 Tekki <in...@tokon.net>:

Have you tried this? http://mojolicio.us/perldoc/Mojo/Content#relaxed

--
You received this message because you are subscribed to a topic in the Google Groups "Mojolicious" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mojolicious/-Ro0_w4cblg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mojolicious...@googlegroups.com.

[sri]

Otherwise you verify the spec and if you find the Mojo::UA behaviour not according to the spec, provide a spec test.

This is actually a really complicated case, and the current Mojo::UserAgent behavior is intentional. Over the years we've encountered quite a few proxy servers that respond to CONNECT requests with responses like "HTTP/1.0 200 OK\x0d\x0a\x0d\x0a" to indicate that the connection has been established. I'm of course open to changing the behavior, but i want a very good explanation for why we should do it (backed up by some serious research).

    https://github.com/kraih/mojo/blob/master/lib/Mojo/Content.pm#L115

Here's the block of code responsible for handling the special case.

--

sebastian

[Ivan Korjavin]

Thank you all.

I not preteding to change behavior due to one buggy server.

But i still searching the way for workaround.

Looks like relaxed is the way. Sorry Tekky, I didn't undersant it well.

This works ok:

my $res = Mojo::Message::Response->new;

$res->content->relaxed(1);

$res->parse("HTTP/1.0 200 OK\x0d\x0a");
$res->parse("Content-Length: 0\x0d\x0a");
$res->parse("Content-Type: text/plain\x0d\x0a\x0d\x0a");
$res->parse('Hello World!');
print $res->to_string;

But I have still problem with UA. Probably my relaxed setting rewrited somewhere. Thats still not work:

            my $tx= $ua->get($uri);                                                                                                                                       
            $tx->res->content->relaxed(1);                                                                                                                                
            my $res=$tx->res;

--
You received this message because you are subscribed to a topic in the Google Groups "Mojolicious" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mojolicious/-Ro0_w4cblg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mojolicious...@googlegroups.com.

[sri]

I'm of course open to changing the behavior, but i want a very good explanation for why we should do it (backed up by some serious research).

For the record, it appears that curl can handle all these special cases, so if anyone knows how we could do the same, i'm all ears.

--

sebastian 

[sri]

For the record, it appears that curl can handle all these special cases, so if anyone knows how we could do the same, i'm all ears.

There seems to be no "correct" way to handle this, since some cases are just plan broken, but i've given a more DWIM solution a try.

    https://github.com/kraih/mojo/commit/079514fe0f0c5b2a614c8f289469d6e684faa81e

--

sebastian

[Ivan Korjavin]

Thank you ALOT!

That solve my problem.

I test it already, and if works fine for me.

--
You received this message because you are subscribed to a topic in the Google Groups "Mojolicious" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mojolicious/-Ro0_w4cblg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mojolicious...@googlegroups.com.