multipart POST syntax question

[Nick Sincaglia]

I am trying to upload a binary file to the Box API. Their documentation provides an example using curl:

curl https://upload.box.com/api/2.0/files/content \
-H "Authorization: Bearer ACCESS_TOKEN" -X POST \
-F attributes='{"name":"tigers.jpeg", "parent":{"id":"11446498"}}' \
-F file=@myfile.jpg

The definition for the ‘-F’ can be found here: http://curl.haxx.se/docs/manpage.html#-F

I was trying to follow this example when I created my code:
http://fgeorges.org/wiki/EXPath#HTTP_Client

In order to try to make this same call using the EXPath HTTP CLient module, I created the following:

let $post-request := (
<http:request xmlns:http="http://expath.org/ns/http-client" href="https://upload.box.com/api/2.0/files/content" method="POST">
    <http:header name="Authorization" value="Bearer ACCESS_TOKEN"/>
    <http:multipart media-type="multipart/form-data" boundary="xyzBouNDarYxyz">
        <http:header name="content-disposition" value="'form-data; name=“attributes"'/>
        <http:body media-type=“text">{{"name":"red-square.png", "parent":{"id”:"12345678"}}}</http:body>
        <http:header name="content-disposition" value="form-data; name=‘“file"; filename="red-square.png"'/>
        <http:body media-type="binary" src="file:/db/tmp/red-square.png"/>
   </http:multipart>
</http:request>)

let $send-request := http:send-request($post-request)
return $send-request

The binary image I am trying to upload is stored inside my eXist-db in the /db/tmp collection. I am getting a http status = 405 from BOX. I suspect I have some syntax error somewhere or I am misinterpreting the samples I have tried to model my code from.

Any help in identifying the issue would be greatly appreciated.

Thanks,

Nick

[Florent Georges]

Hi Nick,

I am travelling now, and can't investigate, but just a few clues...

- 405 is for non-supported method, which does not help really, here
- I find HTTPie <https://github.com/jkbrzt/httpie> an interesting
replacement for CURL
- maybe http://httpbin.org/post can help (see http://httpbin.org/), or
http://requestb.in/
- using a tool like tcpdump
<https://danielmiessler.com/study/tcpdump/> or Wireshark can help a
lot in spotting the differences between both, by looking at what is
actually sent over the wire
- maybe the HTTP Client implementation in eXist provides some logging
with more information about what is sent
- in the first content-disposition, there is an extra " after value=

Please post any more information you can find in the meantime. I'll
try to have a look as soon as I can.

Regards,

--
Florent Georges
http://fgeorges.org/
http://h2oconsulting.be/

> --
> You received this message because you are subscribed to the Google Groups
> "EXPath" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to expath+un...@googlegroups.com.
> To post to this group, send email to exp...@googlegroups.com.
> Visit this group at http://groups.google.com/group/expath.
> For more options, visit https://groups.google.com/d/optout.

[Adam Retter]

Nick,

Often it can help to see the real HTTP data that is being sent by the
EXPath http module. I don't know if you are familiar with Wireshark or
tcpdump, but if so you can use that to capture the real HTTP request.
If you could share that with us, that might help us debug the EXPath
http client.

> --
> You received this message because you are subscribed to the Google Groups
> "EXPath" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to expath+un...@googlegroups.com.
> To post to this group, send email to exp...@googlegroups.com.
> Visit this group at http://groups.google.com/group/expath.
> For more options, visit https://groups.google.com/d/optout.

--
Adam Retter

skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk

[Nick Sincaglia]

Hi Florent,

Before trying to tackle the specific BOX API message we are trying to send, could you provide us with some insights on some of the EXPath HTTP Client settings? I have read through the documentation and looked at a number of code examples but I am still not 100% sure I have my setting set correctly.

Here is the test program I am using:

let $content := util:binary-doc('/db/tmp/red-square.png')

let $post-request := (
    <http:request href="https://upload.box.com/api/2.0/files/content" method='POST'>

        <http:header name="Authorization" value=“Bearer ACCESS_TOKEN">

            <http:multipart media-type="multipart/mixed" boundary="xyzBouNDarYxyz">
                <http:header name="content-disposition" value="attributes"/>
                <http:body media-type="application/json">{'{"name":"red-square.png", "parent":{"id”:"3942954637"}}'}</http:body>
                <http:header name="Content-Type" value="image/jpeg"/>
                <http:body media-type="image/png">{$content}</http:body>
            </http:multipart>
        </http:header>

    </http:request>)

let $send-request := http:send-request($post-request)
return $send-request

We have OAuth2 working with the refresh and access tokens. We know that works.

We are not sure about some of our settings. In particular, we are not sure if our media-type setting is right in this tag:
 <http:multipart media-type="multipart/mixed" boundary="xyzBouNDarYxyz”>

Or if our name/value attributes are correct in this tag:
<http:header name="content-disposition" value="attributes”/>

I think <http:body media-type="image/png">{$content}</http:body> is correct but we are not entirely sure.

Can you first comment on these settings? We are not sure what the allowed values could be for some of these settings as well.

Nick

[Florent Georges]

On 10 August 2015 at 21:56, Nick Sincaglia wrote:

Hi,

> We have OAuth2 working with the refresh and access tokens. We know
> that works.

The first thing you need to know here, is the sequence of HTTP
requests and responses flowing around. The HTTP Client is a low-level
API, not providing you with any OAuth mechanism, for instance.

> We are not sure about some of our settings. In particular, we are
> not sure if our media-type setting is right in this tag:
> <http:multipart media-type="multipart/mixed" boundary="xyzBouNDarYxyz”>

It is... if you want to send a multipart/mixed request, with the given
boundary. I don't know the Box API, but the element looks correct to
me.

> Or if our name/value attributes are correct in this tag:
> <http:header name="content-disposition" value="attributes”/>

That will generate a HTTP header like: "content-disposition:
attributes". Actually, because it is in a multipart request,
technically it is not a HTTP header, it is part of the request entity
body, which itself is a sequence of pseudo-headers + body content
pairs. That is how multipart works:

```
Content-Type: multipart/alternative; boundary=xyzBouNDarYxyz
X-Comment: real headers

--xyzBouNDarYxyz
Content-Type: text/plain
X-Comment: pseudo-headers part #1

Hello, world!
--xyzBouNDarYxyz
Content-Type: application/xml
X-Comment: pseudo-headers part #2

<hello>World!</hello>
--xyzBouNDarYxyz--
```

This request can be represented by:

```
<http:request>
<http:header name="X-Comment" value="real headers"/>
<http:multipart media-type="multipart/alternative" boundary="xyzBouNDarYxyz">
<http:header name="X-Comment" value="pseudo-headers part #1"/>
<http:body media-type="text/plain">Hello, world!</http:body>
<http:header name="X-Comment" value="pseudo-headers part #2"/>
<http:body media-type="application/xml">
<hello>World!</hello>
</http:body>
</http:multipart>
</http:request>
```

As you can see, there is a positional grouping within http:multipart:
the pseudo-headers apply to the next body in the sequence.

> I think <http:body media-type="image/png">{$content}</http:body> is
> correct but we are not entirely sure.

That's an interesting question, as it involves binary data. I am not
sure what the eXist function util:binary-doc() returns, and how it
gets added to the element. What I do with binary (and usually with
body content in general, unless it is static in the request) is to use
the 3d parameter of http:send-request(), namely $bodies. The query:

http:send-request(
<http:request href="...">
<http:body media-type="plain/text">Hello, world!</http:body>
</http:request>)

is the same as:

http:send-request(
<http:request href="...">
<http:body media-type="plain/text"/>
</http:request>,
(),
'Hello, world!')

That is, http:body may serve only as a placeholder for its attributes
values, the content itself being passed as an extra parameter. In
case of a multipart request with, say, 2 http:body elements, you pass
a sequence of 2 items for the parameter. With the above multipart
sequence, if you remove the content of the http:body elements, you can
have instead (showing only the relevant bits):

http:send-request(
<http:request href="...">
<http:multipart media-type="...">
<http:body media-type="plain/text"/>
<http:body media-type="application/xml"/>
</http:multipart>
</http:request>,
(),
('Hello, world!', <hello>World!</hello>))

If util:binary-doc() returns either a xs:hexBinary or a
xs:base64Binary, then you can pass it in the 3 parameter then.

If that does not help you, you will need to describe exactly the HTTP
request you want to send, if you want us to double-check you got the
syntax right. Here I am half guessing, half describing how it works
in general. But that should (hopefully) give you the keys for what
you're trying to do.