problems trying to upload a file
Stjepan Rajko
I recently started working on an Android app that is supposed to upload a file to a user's SoundCloud account. I got the oauth authentication part to work (I end up with an access token & secret), but I got stuck with the actual file upload part. I would appreciate it if someone could share any advice on how to proceed.
(BTW, this is my first time trying to use a web service API, so I might not be saying things 100% accurately)
I currently have a little code snippet that attempts to upload a file by specifying it's title and providing some hard coded asset_data (the content is just "RIFF"). I can sign and POST the request to http://api.soundcloud.com/tracks, but I get as a response Bad Request (400) - Bad Request.
Currently, the entire request is encoded as a x-www-form-urlencoded (because that is all I have figured out how to do so far). Here is an example of its content:
oauth_consumer_key=TU7rx9XbIK0bAoB1Bgqw&oauth_nonce=pibcleskhg75lbearpgl&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1255309328&oauth_token=v4kJxpkzlmrlKjfY6NJjkA&oauth_version=1.0&track%5Basset_data%5D=RIFF&track%5Btitle%5D=Upload&oauth_signature=ecQIW7H5eR693zL95JsYsjtjc1E%3D
Can anyone tell me what exactly is causing the Bad Request response? I suspect that it might have something to do with the fact that I am trying to send a file as x-www-form-urlencoded rather than multipart/form-data. If that's it, how exactly do I format the request, and which parts are used to form the Signature Base String? I saw something potentially helpful here: http://getsatisfaction.com/oauth/topics/can_oauth_be_used_for_file_upload but I don't understand it fully.
Thanks in advance,
Stjepan
Stjepan
http://code.google.com/p/soundclouddroid/source/browse/SoundCloudDroid/src/org/urbanstew/SoundCloudDroid/SoundCloudRequest.java
(specifically, the uploadFile method)
On Oct 11, 6:19 pm, Stjepan Rajko <stjepan.ra...@gmail.com> wrote:
> Hello,
>
> I recently started working on an Android app that is supposed to upload a
> file to a user's SoundCloud account. I got the oauth authentication part to
> work (I end up with an access token & secret), but I got stuck with the
> actual file upload part. I would appreciate it if someone could share any
> advice on how to proceed.
>
> (BTW, this is my first time trying to use a web service API, so I might not
> be saying things 100% accurately)
>
> I currently have a little code snippet that attempts to upload a file by
> specifying it's title and providing some hard coded asset_data (the content
>
> Currently, the entire request is encoded as a x-www-form-urlencoded (because
> that is all I have figured out how to do so far). Here is an example of its
> content:
>
> oauth_consumer_key=TU7rx9XbIK0bAoB1Bgqw&oauth_nonce=pibcleskhg75lbearpgl&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1255309328&oauth_token=v4kJxpkzlmrlKjfY6NJjkA&oauth_version=1.0&track%5Basset_data%5D=RIFF&track%5Btitle%5D=Upload&oauth_signature=ecQIW7H5eR693zL95JsYsjtjc1E%3D
>
> Can anyone tell me what exactly is causing the Bad Request response? I
> suspect that it might have something to do with the fact that I am trying to
> send a file as x-www-form-urlencoded rather than multipart/form-data. If
> that's it, how exactly do I format the request, and which parts are used to
Hannes Tydén
> Currently, the entire request is encoded as a x-www-form-urlencoded (because
> that is all I have figured out how to do so far). Here is an example of its
> content:
>
> oauth_consumer_key=TU7rx9XbIK0bAoB1Bgqw&oauth_nonce=pibcleskhg75lbearpgl&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1255309328&oauth_token=v4kJxpkzlmrlKjfY6NJjkA&oauth_version=1.0&track%5Basset_data%5D=RIFF&track%5Btitle%5D=Upload&oauth_signature=ecQIW7H5eR693zL95JsYsjtjc1E%3D
>
> Can anyone tell me what exactly is causing the Bad Request response? I
> suspect that it might have something to do with the fact that I am trying to
> send a file as x-www-form-urlencoded rather than multipart/form-data. If
> that's it, how exactly do I format the request, and which parts are used to
> form the Signature Base String? I saw something potentially helpful here:
> http://getsatisfaction.com/oauth/topics/can_oauth_be_used_for_file_upload
> but I don't understand it fully.
When creating a track the asset_data is required. This can only be
sent using multipart/form-data. Also the track requires an actual
audio file to successfully create a track. So this is most probably
the source of your error.
If you would have problems with the signing of request you would get
401 Unauthorized errors.
Happy coding!
Cheers,
Hannes
--
Hannes Tydén
Developer, SoundCloud
Mail: han...@soundcloud.com
Skype: hannestyden
Phone: +49 157 7471 6101
Fax: +49 30 8187 8724
Auguststrasse 5a, 101 17, Berlin, DE
What is SoundCloud?
http://soundcloud.com/tour
Stjepan Rajko
When creating a track the asset_data is required. This can only be
sent using multipart/form-data. Also the track requires an actual
audio file to successfully create a track. So this is most probably
the source of your error.
Thank you for your response. I now tried putting together a multipart/form-data request. This is an example of what I get:
--pM0NLn_KNwfwDIwmiInVkceijoJZw8p
Content-Disposition: form-data; name="oauth_consumer_key"
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
TU7rx9XbIK0bAoB1Bgqw
--pM0NLn_KNwfwDIwmiInVkceijoJZw8p
Content-Disposition: form-data; name="oauth_nonce"
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
i4ug2df2b1b307e11943
--pM0NLn_KNwfwDIwmiInVkceijoJZw8p
Content-Disposition: form-data; name="oauth_signature_method"
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
HMAC-SHA1
--pM0NLn_KNwfwDIwmiInVkceijoJZw8p
Content-Disposition: form-data; name="oauth_timestamp"
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
1255535658
--pM0NLn_KNwfwDIwmiInVkceijoJZw8p
Content-Disposition: form-data; name="oauth_token"
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
v4kJxpkzlmrlKjfY6NJjkA
--pM0NLn_KNwfwDIwmiInVkceijoJZw8p
Content-Disposition: form-data; name="oauth_version"
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
1.0
--pM0NLn_KNwfwDIwmiInVkceijoJZw8p
Content-Disposition: form-data; name="track[title]"
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
Upload
--pM0NLn_KNwfwDIwmiInVkceijoJZw8p
Content-Disposition: form-data; name="oauth_signature"
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
bWx5dJgElXDZ4oaAlhMApcG5N6g=
--pM0NLn_KNwfwDIwmiInVkceijoJZw8p
Content-Disposition: form-data; name="track[asset_data]"; filename="audio_1.wav"
Content-Type: application/octet-stream; charset=ISO-8859-1
Content-Transfer-Encoding: binary
....
--pM0NLn_KNwfwDIwmiInVkceijoJZw8p--
And the file is an actual audio file. Does this request appear to be formatted correctly?
If you would have problems with the signing of request you would get
401 Unauthorized errors.
And that's where I am now :-) This is the signature base string I compose for signing:
Signature Base String:
POST&http%3A%2F%2Fapi.soundcloud.com%2Ftracks&oauth_consumer_key%3DTU7rx9XbIK0bAoB1Bgqw%26oauth_nonce%3Di4ug2df2b1b307e11943%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1255535658%26oauth_token%3Dv4kJxpkzlmrlKjfY6NJjkA%26oauth_version%3D1.0%26track%255Btitle%255D%3DUpload
Is that the right content for the signature base string? Is the file name or file content supposed to be accounted for in the signature base string at all?
Happy coding!
Thanks!
Best,
Stjepan
Hannes Tydén
> Thank you for your response. I now tried putting together a
> multipart/form-data request. This is an example of what I get:
>
The parts of the data you sent looks good, but it's hard to say since
the context is important.
>
> Signature Base String:
>
> POST&http%3A%2F%2Fapi.soundcloud.com%2Ftracks&oauth_consumer_key%3DTU7rx9XbIK0bAoB1Bgqw%26oauth_nonce%3Di4ug2df2b1b307e11943%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1255535658%26oauth_token%3Dv4kJxpkzlmrlKjfY6NJjkA%26oauth_version%3D1.0%26track%255Btitle%255D%3DUpload
>
>
> Is that the right content for the signature base string? Is the file name
> or file content supposed to be accounted for in the signature base string at
> all?
I assume that track[title]=Upload is not a part of the query string so
my answer is: no.
The request body should only be included when you do a POST with
content-type application/x-www-form-urlencoded (see
http://oauth.net/core/1.0/#anchor14 9.1.1. Normalize Request
Parameters). So you should leave those parameters out of the signature
base string.
http://gist.github.com/146876 is small ruby script I wrote to verify
signing of requests and also making the process understandable.
I hope this helps.
Stjepan Rajko
>I assume that track[title]=Upload is not a part of the query string so
> Signature Base String:
>
> POST&http%3A%2F%2Fapi.soundcloud.com%2Ftracks&oauth_consumer_key%3DTU7rx9XbIK0bAoB1Bgqw%26oauth_nonce%3Di4ug2df2b1b307e11943%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1255535658%26oauth_token%3Dv4kJxpkzlmrlKjfY6NJjkA%26oauth_version%3D1.0%26track%255Btitle%255D%3DUpload
>
>
> Is that the right content for the signature base string? Is the file name
> or file content supposed to be accounted for in the signature base string at
> all?
my answer is: no.
The request body should only be included when you do a POST with
content-type application/x-www-form-urlencoded (see
http://oauth.net/core/1.0/#anchor14 9.1.1. Normalize Request
Parameters). So you should leave those parameters out of the signature
base string.
http://gist.github.com/146876 is small ruby script I wrote to verify
signing of requests and also making the process understandable.
I hope this helps.
Thank you for your continuing help. I has definitely helped me make some progress, but still no file upload - now I consistently get "Internal Server Error" when I try to upload a file. Although, when I access resources like "/me/", I am able to retrieve them without error.
I found the following content in the OAuth document you referenced (5.2 Consumer Request Parameters):
--
OAuth Protocol Parameters are sent from the Consumer to the Service Provider in one of three methods, in order of decreasing preference:
- In the HTTP Authorization header as defined in OAuth HTTP Authorization Scheme (OAuth HTTP Authorization Scheme).
- As the HTTP POST request body with a content-type of application/x-www-form-urlencoded.
- Added to the URLs in the query part (as defined by [RFC3986] (Berners-Lee, T., “Uniform Resource Identifiers (URI): Generic Syntax,” .) section 3).
I tried passing the request parameters (oauth_*) under the Authorization header (method 1), but that didn't work at all (for file upload or for /me/). Does SoundCould support this method of providing the request parameters?
I also realized that based on 2., placing the request parameters (oauth_*) into the body of the request doesn't help (since the content-type is multipart/form-data). Is that correct?
In any case, I was left with the 3rd method so I stuck the oauth_* parameters into the query. This is what consistently gives me an Internal Server Error for the file upload case.
An example of my query is:
http://api.soundcloud.com/tracks/?oauth_consumer_key=TU7rx9XbIK0bAoB1Bgqw&oauth_nonce=411hbcp0un0517nvrosf&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1255659344&oauth_token=36Svj18kBB3FN1iZWPjng&oauth_version=1.0&oauth_signature=Mk4LqTITD6wdLfweI3nX%2BC2LUQQ%3D
This is the headers&body:
User-Agent: Jakarta Commons-HttpClient/3.1
Content-Type: multipart/form-data; boundary=0ZcjLtQRcUUCUwHoex90z101RJpvRrk
Host: api.soundcloud.com
Content-Length: 52312
--0ZcjLtQRcUUCUwHoex90z101RJpvRrk
Content-Disposition: form-data; name="track[title]"
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
Upload
Content-Disposition: form-data; name="track[asset_data]"; filename="audio_1.wav"
Content-Type: application/octet-stream; charset=ISO-8859-1
Content-Transfer-Encoding: binary
...
The response is "500 - Internal Server Error". I tried with two different files. I also tried a bunch of other unsuccessful shots in the dark.
Do you have any suggestions on what to try? Or, if you could provide me with a complete POST url,headers&body (and the signature base string for that particular request) that results in a successful file upload, I could probably figure out how to replicate the method used on my end.
Thanks and regards,
Stjepan
Hannes Tydén
> Thank you for your continuing help. I has definitely helped me make some
> progress, but still no file upload - now I consistently get "Internal
> Server Error" when I try to upload a file. Although, when I access resources
> like "/me/", I am able to retrieve them without error.
Internal Server Error is of course something we want to avoid.
>
> I tried passing the request parameters (oauth_*) under the Authorization
> header (method 1), but that didn't work at all (for file upload or for
> /me/). Does SoundCould support this method of providing the request
> parameters?
>
> I also realized that based on 2., placing the request parameters (oauth_*)
> into the body of the request doesn't help (since the content-type is
> multipart/form-data). Is that correct?
>
> In any case, I was left with the 3rd method so I stuck the oauth_*
> parameters into the query. This is what consistently gives me an Internal
> Server Error for the file upload case.
We support all three methods of passing the oauth parameters and, as
the spec says, we prefer the first method.
I've successfully uploaded files and below are some data from the request:
********************************************************************************
Signature Base String
********************************************************************************
POST&http%3A%2F%2Fapi.soundcloud.com%2Ftracks&oauth_consumer_key%3DgLnhFeUBnBCZF8a6Ngqq7w%26oauth_nonce%3DyvyM7hwT2zi1qrnddZZzcFlY1YQ02VO3UPofeguuGA%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1255939172%26oauth_token%3DgScOQnEj6UjXqpmR6YqE4A%26oauth_version%3D1.0
********************************************************************************
Authorization Header
********************************************************************************
OAuth oauth_nonce="yvyM7hwT2zi1qrnddZZzcFlY1YQ02VO3UPofeguuGA",
oauth_signature_method="HMAC-SHA1", oauth_timestamp="1255939172",
oauth_consumer_key="gLnhFeUBnBCZF8a6Ngqq7w",
oauth_token="gScOQnEj6UjXqpmR6YqE4A",
oauth_signature="YL2oB1nDNe3q2oTRFJsPePJ9ozM%3D", oauth_version="1.0"
********************************************************************************
Request Body
********************************************************************************
--boundary-one-742652124
content-disposition: form-data; name="track[title]"
This is uploaded from plain.rb
--boundary-one-742652124
content-disposition: form-data; name="track[sharing]"
private
--boundary-one-742652124
content-disposition: form-data; name="track[asset_data]"; filename="knaster.mp3"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
ID3CTT2Vinyl Scratch 07COMengiTunPGAP0TENiTunes v7.4.2COMhengiTunNORM
000000A7 000000A5 00000BD1 00000BA8 00000605 00000605 00007E66
00007EBD 00000068 00000068COM?engiTunSMPB 00000000 00000210 0000089C
00000000000227D4 00000000 00011E88 00000000 00000000 00000000 00000000
00000000 00000000 *** The rest of the file contents are omitted. ***
--boundary-one-436885369--
********************************************************************************
Stjepan Rajko
Internal Server Error is of course something we want to avoid.
...
We support all three methods of passing the oauth parameters and, as
the spec says, we prefer the first method.
I've successfully uploaded files and below are some data from the request:
First, thanks for indicating that the Authorization header should work. I was able to switch both the GET /me and the POST /tracks to use the Authorization header rather than the url/query. However, that did not resolve the Internal Server Error.
Second, thanks for sending the details of a functioning request. It helped me to get around the Internal Server Error, as follows: In my requests, a parameter was transmitted like this in the body:
---
Content-Disposition: form-data; name="track[title]"
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
Upload
I noticed in your body, there was no Content-Type or Content-Transfer-Encoding. I removed those from my string/text parameters to end up with something like this:
---
Content-Disposition: form-data; name="track[title]"
---
And the response: Created!
So... if it should be OK for a consumer to include the Content-Type and Content-Transfer-Encoding (like I was because the http client library I am using was throwing them in there until I took a knife to it), you may want to fix things on your end to avoid the Internal Server Error.
In any case, thanks a ton for helping me out. I think I am good to go now.
Best,
Stjepan
Hannes Tydén
> Second, thanks for sending the details of a functioning request. It helped
> me to get around the Internal Server Error, as follows: In my requests, a
> parameter was transmitted like this in the body:
>
> ---
> Content-Disposition: form-data; name="track[title]"
> Content-Type: text/plain; charset=US-ASCII
> Content-Transfer-Encoding: 8bit
>
> Upload
> ---
>
> I noticed in your body, there was no Content-Type or
> Content-Transfer-Encoding. I removed those from my string/text parameters
> to end up with something like this:
> ---
> Content-Disposition: form-data; name="track[title]"
>
> Upload
> ---
>
> And the response: Created!
>
> So... if it should be OK for a consumer to include the Content-Type and
> Content-Transfer-Encoding (like I was because the http client library I am
> using was throwing them in there until I took a knife to it), you may want
> to fix things on your end to avoid the Internal Server Error.
Thank you for finding this, it seems like a "low level" error and I'll
need to assess what is expected of clients and servers when doing
multipart uploads.
Good look with the project!
- Hannes
josefsson
Request.
I believe I do everything as you described above and here is the post
request:
------------8cc30e57ba4771e
Content-Disposition: form-data; name="track[asset_data]";
filename="test.mp3"
Content-Type: application/octet-stream
*omitted file data*
------------8cc30e57ba4771e
Content-Disposition: form-data; name="track[title]";
Test
------------8cc30e57ba4771e
Content-Disposition: form-data; name="track[sharing]";
private
------------8cc30e57ba4771e--
The OAuth parameters are sent in the query, I believe them to be
correct as I do not get a 401 error?
--
Anders Josefsson
On 20 Okt, 17:27, Hannes Tydén <han...@soundcloud.com> wrote:
> On Tue, Oct 20, 2009 at 6:24 AM, Stjepan Rajko <stjepan.ra...@gmail.com> wrote:
> > Second, thanks for sending the details of a functioningrequest. It helped
Stjepan Rajko
Content-Disposition headers?
E.g., instead of
Content-Disposition: form-data; name="track[asset_data]"
semicolon at the end.
Best,
Stjepan
josefsson
It does, however, not solve the problem. You are correct that the ;
should be removed at the end of the Content-Diposition values, but
values should be seperated by it, so it is correct that there is a ;
after track[asset_data] but it should be removed after track[title]
and track[sharing].
On 11 Nov, 17:01, Stjepan Rajko <stjepan.ra...@gmail.com> wrote:
> Does it help to take off the semicolons at the end of your
> Content-Disposition headers?
>
> E.g., instead of
> Content-Disposition: form-data; name="track[asset_data]";
>
> do
>
> Content-Disposition: form-data; name="track[asset_data]"
>
>
> Best,
>
> Stjepan
>
josefsson
sample by Hannes, and now I get Internal Server Error instead.
Here is a TCP-dump of it: http://josefsson.playven.com/soundcloud.dump
It includes all the data that is being sent and recived.
Stjepan Rajko
I'v removed the ; and changed the order to the same as the above
sample by Hannes, and now I get Internal Server Error instead.
Here is a TCP-dump of it: http://josefsson.playven.com/soundcloud.dump
It includes all the data that is being sent and recived.
Try putting a blank line before the file content. E.g., instead of:
Content-Disposition: form-data; name="track[asset_data]"; filename="test.mp3"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
ID3...
Try:
Content-Disposition: form-data; name="track[asset_data]"; filename="test.mp3"
Content-Type: application/octet-stream
ID3...
Does that help?
Best,
Stjepan