Does Mojolicious support post/put with JSON arguments?

[Joe Landman]

Looking to craft a very simple example for this and I am running into a
problem. Using a simple post method for this


$thingy = MyObject->new;

post '/' => sub {
my $self = shift;
# ...
my @params = $self->param;
my $content;

# ...
map { $content->{$_} = $self->param($_) } @params;
# $content should be a hashref pointing to the parameters in a
# simple key => value store.

# ...
# pass $content to MyObject for processing
$result = $thingy->method($content);
}

Can we pass the parameters to this post as a JSON object
(application/json) versus a simple "form" object?


Thanks!

Joe
---
joe.l...@gmail.com

[Matt Price]

Of course!  Just do the following:

$self->req->json

Returns a hashref.

-Matt

[Joe Landman]

On 09/15/2013 01:33 PM, Matt Price wrote:
> Of course! Just do the following:
>
> $self->req->json
>
> Returns a hashref.

Thanks ... I think I've been beating my head against either pilot error
or a bug then. I can't figure out which at the moment.

I was using Mojolicious::UserAgent to try to do the post, and no
parameters were getting through at all. Then, for laughs, I switched to
LWP::UserAgent, and it worked.


Server:

#!/usr/bin/perl -w
use strict;

use Mojolicious::Lite;
use JSON;
use XML::Simple;
use Data::Dumper;

post '/' => sub {
my $self = shift;

my @params = $self->param;
my $content;

my ($output,$status,$json,$xml,$txt);

$json = JSON->new->allow_nonref;

# copy parameters into content hash
foreach my $p (@params) {
next if ($p =~ /_type/i);
$content->{$p} = $self->param($p);
}

my $text = "ok: ";
map {$text .= (sprintf "%s=%s, ",$_,$content->{$_})} @params;
$self->render(text => $text);

};

app->start(qw(daemon --listen http://*:3000));


Mojolicious::UserAgent client (call this q.pl)


#!/usr/bin/perl
use strict;
use Mojo::UserAgent;
use Data::Dumper;

my ($res,$tx,$headers,$ua,$t);
$t = Mojo::UserAgent->new;
$tx = $t->post('http://localhost:3000/' => json => { a => '1', b => '2'});
$res = $tx->res->body;
printf "result = %s\n ",Dumper($res);




LWP::UserAgent version (call this p.pl)

#!/usr/bin/perl
use strict;
use LWP::UserAgent;
use Data::Dumper;

my ($res,$tx,$headers,$ua,$t);
$t = LWP::UserAgent->new;
$tx = $t->post('http://localhost:3000/' , { a => '1', b => '2'});
$res = $tx->content;
printf "result = %s\n ",Dumper($res);


Run the server, and then first the q.pl code (Mojolicious::UserAgent)

landman@lightning:~$ ./q.pl
result = $VAR1 = 'ok: ';


then the p.pl code (LWP::UserAgent)

landman@lightning:~$ ./p.pl
result = $VAR1 = 'ok: a=1, b=2, ';


But if I change the q.pl to r.pl by using

$tx = $t->post('http://localhost:3000/' => form => { a => '1', b => '2'});

I get the expected result:

landman@lightning:~$ ./r.pl
result = $VAR1 = 'ok: a=1, b=2, ';


Basically I am not sure if this is pilot error, a bug, or a
documentation ambiguity (c.f.
http://search.cpan.org/~sri/Mojolicious-4.37/lib/Mojo/UserAgent.pm#post
where the form

my $tx = $ua->post(
'http://example.com' => {DNT => 1} => json => {a => 'b'});

is specifically indicated).

I was thinking this was a server side issue, but I am wondering if its
really a client side Mojo::UserAgent bug in docs or code ...). Still
could be pilot error, and I'd be happier if it were.

Joe

--
joe.l...@gmail.com

[Matt Price]

Ahh I think the issue is how you are trying to decode the message for your Mojo UserAgent.  Trying adding:
use Mojo::JSON;

my $json = Mojo::JSON->new;

and then change your $res to:
$res        = $json->decode($tx->res->body);

[Joe Landman]

The TL;DR version is, no, it does not appear that Mojolicious enables
you to pull in json parameters via params. But, they are there under

$self->{tx}->{req}->{content}->{asset}->{content}

in a nice stringified manner, perfect for turning into a hash.

Longer:

Ok, so I pulled out my handy Komodo debugger and started single stepping
through the server side once it got into the post '/'. I had to dig
through the data structure of the request object to see it.

Watching this content when I run the original form submission through, I
see this value in there:

a=1&b=2

with normal form submission, and

{"a":"1","b":"2"}

with json form submission.

From this I surmise that the client libraries (Mojolicious::UserAgent)
used in this example is fine. And that there might be an oddity in how
that content gets parsed on the server ... in that json content appears
to be thrown away.

I stepped through the Mojo::Message module, and noted the body_params
method was used to walk through the parameters and bring them into the
params object. There are parsers for
"application/x-www-form-urlencoded" and "multipart/form-data". There is
no parser for application/json.

There is also a json method, but it isn't in this pathway.

I tried adding a little logic to the body_params method to parse the
JSON data (should be as easy as a call to Mojo::JSON->decode($json), and
then populating a simple array. I wasn't able to figure out where the
$json string was stored though.

I've got a temporary workaround (as above) but I think longer term and
far cleaner would be to pull the JSON form submission through the param
method. If I have time to work through a patch, I send one along, but
if someone else solves this, or has a better workaround in short order,
please do post it.


Joe

--
joe.l...@gmail.com

[Sebastian Riedel]

> I've got a temporary workaround (as above) but I think longer term and far cleaner would be to pull the JSON form submission through the param method. If I have time to work through a patch, I send one along, but if someone else solves this, or has a better workaround in short order, please do post it.

What if the JSON content is an array with uneven number of values or a deeply nested structure? The whole idea of treating JSON like form values makes no sense to me.

--
Sebastian Riedel
http://github.com/kraih
http://twitter.com/kraih
http://mojolicio.us

[Joe Landman]

On 09/15/2013 06:41 PM, Sebastian Riedel wrote:
>> I've got a temporary workaround (as above) but I think longer term
>> and far cleaner would be to pull the JSON form submission through
>> the param method. If I have time to work through a patch, I send
>> one along, but if someone else solves this, or has a better
>> workaround in short order, please do post it.
>
>
> What if the JSON content is an array with uneven number of values or
> a deeply nested structure? The whole idea of treating JSON like form
> values makes no sense to me.

Ok ... is it easier to just present it up raw (in stringified form) to
be converted with Mojo::JSON? I don't want to hard wire in a mechanism
that might disappear in future releases ... I had thought param was the
cleaner way to do it.

JSON is inherently a key value store though, so it really does have a 1
to 1 mapping with a perl hash. If there is an incorrect JSON format
handed in, it is possible to make a parser very unhappy.

Param is inherently a flat key-value store. So its going to miss the
nesting aspect of JSON.

Not complaining, just exploring how to do this. Basically, I want the
post/put to accept pretty much anything we can throw at it that "makes
sense" for an appropriate value of "makes sense".

Joe

--
joe.l...@gmail.com

[Sebastian Riedel]

> I don't want to hard wire in a mechanism that might disappear in future releases …

What?

> JSON is inherently a key value store though…

Not at all.

[sri]

> JSON is inherently a key value store though…

Not at all.

This test has many fun examples for what valid JSON can look like.

    https://github.com/kraih/mojo/blob/master/t/mojo/json.t

--

sebastian 

[Joe Landman]

On 09/15/2013 06:58 PM, Sebastian Riedel wrote:
>> I don't want to hard wire in a mechanism that might disappear in future releases �
>
> What?

This is a hack around the problem. If I use this, the big long nasty
json_arg_string = ... could be made to go away as the software updates.
That is, I am reaching into inner workings of the object, and I am
generally uncomfortable doing so, as you and the Mojolicious team could
chose to make changes in the future.

sub get_any_arg_types_into_hash {
my $self = shift;

my @params = $self->param;

my $json_arg_string =
$self->{tx}->{req}->{content}->{asset}->{content};
my $content;

# copy parameters into content hash

if (@params) {
foreach my $p (@params) {

$content->{$p} = $self->param($p);
}
}

else
{
$content = Mojo::JSON->decode($json_arg_string);
}
return $content;
}

Then I get my parameters in a nice easy to use hash, regardless of
whether they came in via parameters (http form) or the
content->asset->content bit (json). I really don't like the latter for
the previously mentioned reason, but I can use it and change it easily
later. That is the hardcoding I am worried about.

I'll have to see if Mojo::JSON handles more complex objects (I am not
completely sure what we will be receiving from the client code, so I
need a more flexibility), but we can work with this for now, and fix it
later if a better/saner/less dangerous mechanism emerges.


Joe

--
joe.l...@gmail.com

[Sebastian Riedel]

> my $json_arg_string = $self->{tx}->{req}->{content}->{asset}->{content};

This is madness, don't ever do that.

my $json_arg_string = $self->req->body;
my $json = $self->req->json;

[Joe Landman]

On 09/15/2013 08:30 PM, Sebastian Riedel wrote:
>> my $json_arg_string = $self->{tx}->{req}->{content}->{asset}->{content};
>
> This is madness, don't ever do that.

This is precisely why I asked, and probed what was going on. I
didn't/don't like it.

>
> my $json_arg_string = $self->req->body;
> my $json = $self->req->json;

Thanks, will try this out.

[Sebastian Riedel]

> This is precisely why I asked, and probed what was going on.

I've now added an example early in the tutorial, hopefully no beginner will stumble over this again.

https://github.com/kraih/mojo/commit/eeac9ab0ff88326310b03a9ac277e1b43bb1635f

[Joe Landman]

On 09/15/2013 08:41 PM, Sebastian Riedel wrote:
>> This is precisely why I asked, and probed what was going on.
>
>
> I've now added an example early in the tutorial, hopefully no beginner will stumble over this again.
>
> https://github.com/kraih/mojo/commit/eeac9ab0ff88326310b03a9ac277e1b43bb1635f

I and many other veterans (and beginners) appreciate this. Thanks.


Joe