Uploading files to google doc using cURL

[laraaj]

Can anyone please help me how i can upload files using cURL i can connect successfully to google doc and google spreadsheets using cURL but still

can't manage to figure out how to upload files. according to this post on blogger http://googleappsdeveloper.blogspot.com/2011/05/upload-all-file-types-to-any-google.html google now allows uploading files using any account.

i use curl to get an authentication code like this :

writelyAuth=`curl -Ss https://www.google.com/accounts/ClientLogin $postrquest -d service=writely -k | grep "Auth=" | sed s/"Auth="/"auth="/`

where $postrquest variable is  :

 "-d Email=mye...@gmail.com -d Passwd=mypassword -d accountType=GOOGLE -d source=Google"

and then i request forlders list like this :

curl -Ss -o $folderxml -k -H "Authorization: GoogleLogin $writelyAuth" -H "GData-Version: 3.0" "https://docs.google.com/feeds/default/private/full/folder%3Aroot/contents/-/folder"

 

i tried this : 

curl  -k -T "/root/Spice Girls Viva Forever.mp3" -H "Content-Length: 4950314" -H "Authorization: GoogleLogi -H "GData-Version: 3.0" -H "Content-Type: audio/mpeg" -H "Slug: My Experiment With Hip Hop" "https://docs.google.com/feeds/default/private/full/?convert=false"

but it didn't work

thanks in advance.

[Ali Afshar]

Hi Laraaj,

I'm not sure curl is the best tool to use here, as it can get a bit complicated with all those parameters and escaping, but can you turn on curl tracing so we can see what is going on, by adding something like:

--trace-ascii /dev/stdout

Thanks

--
Ali Afshar | www.googplus.org/ali | Google Developer Relations

[laraaj]

Hi, thanks for your replay actually there is no escaping needed for
this and the Curl manual is quite clear about all parameters.
the problem is with the google API that's not clear and it seems like
google don't support using it with tools like curl.

here is the curl command i used this time:

curl -k -T "/root/database.sql.gz" -H "Content-Length: 1868" -H
"Authorization: GoogleL: 3.0" -H "Content-Type: audio/mpeg" -H
"Slug:mydata" --trace-ascii /dev/stdout "https://docs.google.com/
feeds/default/private/full/?convert=false"

and here is the out put :

curl -k -T "/root/database.sql.gz" -H "Content-Length: 1868" -H
"Authorization: GoogleLogi $writelyAuth" -H "GData-Version: 3.0" -H
"Content-Type: audio/mpeg" -H "Slug:mydata" "https://docs.google.com/
feeds/default/private/full/?convert=false"
<errors xmlns='http://schemas.google.com/g/2005'><error><domain>GData</
domain><code>invalidRequestUri</code><internalReason>Invalid request
URI</internalReason></error></errors>root@Laraajnixpc:~# curl -k -T "/
root/database.sql.gz" -H "Content-Length: 1868" -H "Authorization:
GoogleL: 3.0" -H "Content-Type: audio/mpeg" -H "Slug:mydata" --trace-
ascii /dev/stdout "https://docs.google.com/feeds/default/private/
full/?convert=false"
== Info: About to connect() to docs.google.com port 443 (#0)
== Info: Trying 74.125.39.113... == Info: connected
== Info: Connected to docs.google.com (74.125.39.113) port 443 (#0)
== Info: successfully set certificate verify locations:
== Info: CAfile: /usr/share/curl/ca-bundle.crt
CApath: none
== Info: SSLv3, TLS handshake, Client hello (1):
=> Send SSL data, 109 bytes (0x6d)
0000: ...i..No...].s...A.....V....6.Y...5....(.9.8.5.......3.2./.....
0040: ..............................docs.google.com
== Info: SSLv3, TLS handshake, Server hello (2):
<= Recv SSL data, 85 bytes (0x55)
0000: ...Q..No..../.J.>.>.M....QM....y...w.. ...o&..m'..9...mJ.5A+...q
0040: ..^..M#..............
== Info: SSLv3, TLS handshake, CERT (11):
<= Recv SSL data, 1997 bytes (0x7cd)
0000: ..........0...0..q.......Q........t0...*.H........0F1.0...U....U
0040: S1.0...U....Google Inc1"0 ..U....Google Internet Authority0...11
0080: 0812034902Z..120812035902Z0f1.0...U....US1.0...U....California1.
00c0: 0...U....Mountain View1.0...U....Google Inc1.0...U....*.google.c
0100: om0..0...*.H............0.......9...J.>....4K...l..a.,U.....#.y.
0140: P.G.C...fxW../b.l......=A.O.....z..ejP].7..!...NT.a.
0176: 3}..<...L..L9...@L`......m.....#..>4l4.D..........0...0...U.....
01b6: .r.......b.......:...0...U.#..0.....0..C.>g.....j..k.$0[..U...T0
01f6: R0P.N.L.Jhttp://www.gstatic.com/GoogleInternetAuthority/GoogleIn
0236: ternetAuthority.crl0f..+........Z0X0V..+.....0..Jhttp://www.gsta
0276: tic.com/GoogleInternetAuthority/GoogleInternetAuthority.crt0!..+
02b6: .....7.......W.e.b.S.e.r.v.e.r0.....U......0.....*.google.com..g
02f6: oogle.com..*.atggl.com..*.youtube.com..youtube.com..*.ytimg.com.
0336: .*.google.com.br..*.google.co.in..*.google.es..*.google.co.uk..*
0376: .google.ca..*.google.fr..*.google.pt..*.google.it..*.google.de..
03b6: *.google.cl..*.google.pl..*.google.nl..*.google.com.au..*.google
03f6: .co.jp..*.google.hu..*.google.com.mx..*.google.com.ar..*.google.
0436: com.co..*.google.com.vn..*.google.com.tr..*.android.com..*.googl
0476: ecommerce.com0...*.H............".. ]K.....?[`.f...).s...t..mT.f
04b6: ..-.x.>s..OS.[..D.Y$~..cW.H!.|F&..nCn..2.D..DT..?.*.H^...=..*.r.
04f6: .z.....3... ....Q*.eaH.>..g.+..A...0...0...........gq0...*.H....
0536: ....0N1.0...U....US1.0...U....Equifax1-0+..U...$Equifax Secure C
0576: ertificate Authority0...090608204327Z..130607194327Z0F1.0...U...
05b6: .US1.0...U....Google Inc1"0 ..U....Google Internet Authority0..0
05f6: ...*.H............0............W..>@}....qcS..yt..D.3 ..{.(...lI
0636: ~._4YN.z0....WE....s&..2.....;....6.oQD... Rs.&.5j....Y.+.g.9...
0676: .l..%..3.G...Py..i.....Wo........0..0...U...........0...U.......
06b6: .0..C.>g.....j..k.$0...U.#..0...H.h.+....G.# .O3....0...U.......
06f6: 0.......0:..U...3010/.-.+.)http://crl.geotrust.com/crls/secureca
0736: .crl0...*.H..............#.H...|`w^...............f.I....)...$L.
0776: .K.^c..'..6!,..`g......q..Z.'.........!.R..Z.......~x....\L..\U
07b6: )..QB.:.8.x;.........P2
== Info: SSLv3, TLS handshake, Server finished (14):
<= Recv SSL data, 4 bytes (0x4)
0000: ....
== Info: SSLv3, TLS handshake, Client key exchange (16):
=> Send SSL data, 134 bytes (0x86)
0000: .......n<..;..{{.,........f..?s...,*B.}..G3o..t.t{Kz<.......7...
0040: .z.T..X....._^o3.j.Za.....V|....)af..........V..."...y.'i.@.....
0080: ...HE.
== Info: SSLv3, TLS change cipher, Client hello (1):
=> Send SSL data, 1 bytes (0x1)
0000: .
== Info: SSLv3, TLS handshake, Finished (20):
=> Send SSL data, 16 bytes (0x10)
0000: ......a.HO.(u..o
== Info: SSLv3, TLS change cipher, Client hello (1):
<= Recv SSL data, 1 bytes (0x1)
0000: .
== Info: SSLv3, TLS handshake, Finished (20):
<= Recv SSL data, 16 bytes (0x10)
0000: ....T)Z.EJg.<J..
== Info: SSL connection using RC4-SHA
== Info: Server certificate:
== Info: subject: C=US; ST=California; L=Mountain View;
O=Google Inc; CN=*.google.com
== Info: start date: 2011-08-12 03:49:02 GMT
== Info: expire date: 2012-08-12 03:59:02 GMT
== Info: subjectAltName: docs.google.com matched
== Info: issuer: C=US; O=Google Inc; CN=Google Internet
Authority
== Info: SSL certificate verify ok.
=> Send header, 625 bytes (0x271)
0000: PUT /feeds/default/private/full/?convert=false HTTP/1.1
0039: User-Agent: curl/7.21.4 (x86_64-unknown-linux-gnu) libcurl/7.21.
0079: 4 OpenSSL/0.9.8n zlib/1.2.5 libidn/1.22
00a2: Host: docs.google.com
00b9: Accept: */*
00c6: Content-Length: 1868
00dc: Authorization: GoogleLogi auth=DQAAALoAAAAjirYxC5wK3KMO-GVC1SUlG
011c: tD6UAB6GMjGbBt4N4hlKm-Be6-AOt1AjuEav_Retn-66Pj26k-nxlHHZ10ElrkvV
015c: 0-JvYGsgwCvSbiRsMW442W65NjJINMXlGpGSmj55GRmho0jjaVX7DDLZ-P9Ucv50
019c: LKdSyNZe9QUkGT6hL5CKkM_628Iym-6bjjYpVXn28-UHlGLogDkD6cNiC005_8Wx
01dc: kMUdHk1nYcwmAp1uE1I0-XrYWyXOYqVpSS-iw1GYCA
0208: GData-Version: 3.0
021c: Content-Type: audio/mpeg
0236: Slug:mydata
0243: Content-Length: 1868
0259: Expect: 100-continue
026f:
== Info: Done waiting for 100-continue
=> Send data, 1868 bytes (0x74c)
0000: ......aN....ks.F.s.+.....9...|..P.....a....Y....A.H....Wk...\.i.
0040: =.lI...ZI.e.n..G......E.?."Bc+.}k.....3h...#..-..m....k7B.V./...
0080: ..E.zA(#.3.....gLB.$...ch?..zj...>._`y...Nm...../;<.1...].......
00c0: |8....H..t.....e...y.......Lu.&"...1.N..d.......LD...A.^+...U..&
0100: .......e.* 2...9..5..<..<<...n........>.....*7.{............K..[
0140: ....cFp...l....[(#r.l1..g...S.....2....qE.:..-.......(.:.(.}....
0180: ..=..5..8...d4W....C}..PS.<R.D,..{.....D..7oK=Q:H....C. ..U..MB.
01c0: ..&7 ..7SN'K."....#......[....1....-.....4.*L.D.2[..........m1..
0200: ....46.'FH.E@.]cm....roN..hT.... tk...r.S..b+.7.).gy.B.k........
0240: .........0c...$...|.Vo..3..we....xe..)....!E..@.]?N..uE...4.`..d
0280: ...J.....C2..52......l.!.geb......b.\n8en.d.1..Qa.IJ...2..D.g..8
02c0: .4t.[\L.Z.....H...>..k.V..;. .N...=i)Rq`.+.,....IQ.*.......+b.T.
0300: ....(1.z\........e...-.J....[......qV..O..W...#...l...S.....=_..
0340: .......<c|u..4.4.np.{bG.%2:.........fn..~.....s.Bj7.... \.<k..-.
0380: .:...P....=Fw....N.[!......s...!.K.vo... ...C\....%...-n..wM...*
03c0: ..3q..B^..bS..^sr..L(.K.....b.....i.i......JD..fz@.Fe.......2...
0400: ....`..7.....!.]....Od.....s]j....._\t4..K.T..>.vdV3.k..r.n..Q.z
0440: N...*.6%n..<1nh/..{9.x...$I.z.GLP...+.....9F..<@k../jEDJ.z~ur..V
0480: R...U.....=....2M&.....m.w.....B..A......... g....9...s......[..
04c0: ....l.:.....@..z..?.....l. ......-......../..#sy0U3.W..L.\+5...
0500: ..lm....1.6..'.lS.mN.O..zZ5<............_.^T.S...e..B1m.X..a.G.7
0540: .V..Y.J..|.x...........l.....o..........<...d.] c....._..d.bB..)
0580: &...b....X7..)&...b..... .....y<.%........9Z.BF.m..."..,....n..
05c0: H.2.fM...J..6.5.& .......%..).o..tW..kJ'M...E.;.f.....{...(...3)
0600: t.F..N...H.......s2...b.N!...W.c.\..|^...L!...iR<3...L.oq....W..
0640: ....C....K..5..Ni.......J..l.S....^k$.?.X(-{.]...S6......../...
0680: .mm}..sA..x!+....<~(..b.c.+....."Pm2.3`7.a..=....."..`..).+.Cl.
06c0: .l..".7f..RI..s...fGQhP3m..j..,.l.....O..j.TG........w.aw.!x.o..
0700: .v._.'6....m.C..7..... ..}.}o:.O.........$.. ...MCj".E.w.X..D.r.
0740: ?8...m..."..
<= Recv header, 26 bytes (0x1a)
0000: HTTP/1.1 400 Bad Request
<= Recv header, 54 bytes (0x36)
0000: Content-Type: application/vnd.google.gdata.error+xml
<= Recv header, 37 bytes (0x25)
0000: Date: Tue, 13 Sep 2011 17:07:21 GMT
<= Recv header, 40 bytes (0x28)
0000: Expires: Tue, 13 Sep 2011 17:07:21 GMT
<= Recv header, 35 bytes (0x23)
0000: Cache-Control: private, max-age=0
<= Recv header, 33 bytes (0x21)
0000: X-Content-Type-Options: nosniff
<= Recv header, 29 bytes (0x1d)
0000: X-Frame-Options: SAMEORIGIN
<= Recv header, 33 bytes (0x21)
0000: X-XSS-Protection: 1; mode=block
<= Recv header, 13 bytes (0xd)
0000: Server: GSE
<= Recv header, 28 bytes (0x1c)
0000: Transfer-Encoding: chunked
<= Recv header, 2 bytes (0x2)
0000:
<= Recv data, 4 bytes (0x4)
0000: b1
<= Recv data, 177 bytes (0xb1)
0000: <errors xmlns='http://schemas.google.com/g/2005'><error><domain>
0040: GData</domain><code>invalidRequestUri</code><internalReason>Inva
0080: lid request URI</internalReason></error></errors>
<= Recv data, 7 bytes (0x7)
0000:
0002: 0
0005:
== Info: Connection #0 to host docs.google.com left intact
== Info: Closing connection #0
== Info: SSLv3, TLS alert, Client hello (1):
=> Send SSL data, 2 bytes (0x2)
0000: ..
<errors xmlns='http://schemas.google.com/g/2005'><error><domain>GData</
domain><code>invalidRequestUri</code><internalReason>Invalid request
URI</internalReason></error></errors>

[Ali Afshar]

Hi Laraaj,

Thanks for the detailed info. In general if you can do it with a
client library you can also do it with curl, it is just HTTPS after
all, so there would be no way to disallow using curl specifically.

The issues I can see with your request are:

1. Currently if you are uploading a file with convert=false you must
use Resumable Upload [1]. I believe this is not entirely clear in our
documentation, so we are fixing that. Resumable Upload should work
fine with curl, but again will be a bit tricky.

2. You are performing a PUT, which should be a POST (even when you
switch Resumable Upload).

3. Your Authorization header contains "GoogleLogi" instead of "GoogleLogin".

Regards

[1] http://code.google.com/apis/documents/docs/3.0/developers_guide_protocol.html#ResumableUpload

--

[Jim Trimble]

Hi Ali,

With Uploads, I use cURL to upload, but I am getting a failure response saying I don't have permission to do it. All the other functions that use cURL work. My question is, can the upload feature work without having google premier apps or an upgrade storage package? A basic gmail account? Thanks!

[Vic Fryzel]

Jim,

Can I see your cURL command?  I bet it's that you're not using resumable upload to upload the file (which, BTW, is really difficult to do from cURL).  If you can, you should just use our Python client library, which makes this a one liner.

-Vic

[Jim Trimble]

Contained within a class, Also I have tried "@" . $file for POSTFIELDS. Unfamiliar with Python.

$headers = array(

"Authorization: GoogleLogin auth=" . $this->auth,

"GData-Version: 3.0",

"Content-Length: ". filesize($file),

"Content-Type: ". $this->returnMIMEType($file),

"Slug: ". $name);

if($idFolder)

{

$url = "http://docs.google.com/feeds/".$this->username."/private/full/folder%3A". $idFolder ."/contents?convert=false";

}

else

{

$url = "http://docs.google.com/feeds/".$this->username."/private/full?convert=false";

}

// Make the request

curl_setopt($this->curl, CURLOPT_URL, $url);

curl_setopt($this->curl, CURLOPT_HTTPHEADER, $headers);

                        /*curl_setopt($this->curl, CURLOPT_RETURNTRANSFER, 1);*/

curl_setopt($this->curl, CURLOPT_POST, true);

curl_setopt($this->curl, CURLOPT_POSTFIELDS, file_get_contents($file));

curl_setopt($this->curl, CURLOPT_VERBOSE, true);

// Execute

$response = curl_exec($this->curl);

// Get error

if(curl_errno($this->curl)) 

throw new Exception(curl_error($this->curl));

// Parse the response

$response = simplexml_load_string($response);

return $response;

[laraaj]

thanks alot Ali Afshar for your help.

here is how i solved the problem based on this tutorial  http://code.google.com/apis/gdata/articles/using_cURL.html and the precious help of Ali Afshar. 

i used this command to request an upload link:

databaseuploadlink=`curl  -Ss -k --request POST  -H "Content-Length: 0" -H "Authorization: GoogleLogin $writelyAuth" -H "GData-Version: 3.0" -H "Content-Type: application/x-gzip" -H "Slug: $sqlnewdatabase.tar.bz.gpg"  "https://docs.google.com/feeds/upload/create-session/default/private/full?convert=false" -D /dev/stdout | grep "Location:" | sed s/"Location: "//`

which returns a variable with the upload link then i use this one for the actual upload :

curl -Ss -k --request POST --data-binary "@$tmpdir/$sqlnewdatabase.tar.bz.gpg" -H "Content-Length: $databaselength" -H "Authorization: GoogleLogin $writelyAuth" -H "GData-Version: 3.0" -H "Content-Type: application/x-gzip" -H "Slug: $sqlnewdatabase.tar.bz.gpg"  "$databaseuploadlink"

now i'm trying to download the same file using cURL but i'm stuck in a loop of two URLs.

this one which is the actual download link from the xml file of the contents of the folder that contain the file

https://doc-04-c4-docs.googleusercontent.com/docs/securesc/87tbce0piv90fsrmmrb8btavbgp6l1c2/f8l8ip89iapa7117m5bsi9k760jkefao/1316205000000/08304268904738079656/08304268904738079656/0B7gWftMFHKHpZTFlOWFkY2ItMzc0Yy00ZTU2LWJhZmUtNjkzYzg0ZmQwYjQz?h=16653014193614665626&amp;e=download&amp;gd=true

when i request that url it returns a header redirection to this other URL 

https://docs.google.com/nonceSigner?nonce=7chc55448pt1c&continue=https://doc-04-c4-docs.googleusercontent.com/docs/securesc/87tbce0piv90fsrmmrb8btavbgp6l1c2/f8l8ip89iapa7117m5bsi9k760jkefao/1316205000000/08304268904738079656/08304268904738079656/0B7gWftMFHKHpZTFlOWFk'2ItMzc0Yy00ZTU2LWJhZmUtNjkzYzg0ZmQwYjQz?h%3D16653014193614665626%26amp;e%3Ddownload%26amp;gd%3Dtrue&hash=44iqvkt5die5us8rvdi2r74anrq7epsb

if i requested that URL it returns an HTLM file like this one 

<HTML>

<HEAD>

<TITLE>Moved Temporarily</TITLE>

</HEAD>

<BODY BGCOLOR="#FFFFFF" TEXT="#000000">

<H1>Moved Temporarily</H1>

The document has moved <A HREF="https://doc-04-c4-docs.googleusercontent.com/docs/securesc/87tbce0piv90fsrmmrb8btavbgp6l1c2/th0lvvimjvo1kup3mnq542lmo7kesf3j/1316194200000/08304268904738079656/08304268904738079656/0B7gWftMFHKHpZTFlOWFkY2ItMzc0Yy00ZTU2LWJhZmUtNjkzYzg0ZmQwYjQz?h=16653014193614665626&amp;amp;e=download&amp;amp;gd=true&amp;nonce=okkv0p8f4130u&amp;user=08304268904738079656&amp;hash=beh2e78lfurrg00mu6e387rqavrpofqc">here</A>.

</BODY>

</HTML>

here is the the commands i use :

curl  -k  -o "$tmpdir/$sqlolddatabase" -H "Authorization: GoogleLogin $writelyAuth" -H "GData-Version: 3.0"  "$downloadlink"

"$tmpdir/$sqlolddatabase" : is the output file variable 

$writelyAuth : authentication key for google doc using clientlogin method 

"$downloadlink" is the download link variable one of the above links 

anyone has any ideas what maybe the problem i suspect it maybe the scope of the clientlogin which was meant for writely service 

which means this scope https://docs.google.com/feeds/ i couldn't find a service to generate a client login for the scope https://docs.googleusercontent.com/

[Jim Trimble]

Vic do you have a solution using cURL?

On Thu, Sep 15, 2011 at 10:21 AM, Vic Fryzel <vicf...@google.com> wrote:

[Vic Fryzel]

Yep, it's because you're not using resumable upload.  Unfortunately, the PHP client library does not support resumable upload :(  Sorry that I didn't realize sooner you were using PHP.  I thought you were using command-line curl.

If you'd like a Python sample, let me know.  Resumable upload is built in to the Python library, and thus just calling create() will take care of resumable upload for you.

-Vic