Upload issue with chunk buffer size

[Jerry]

I am having a strange issue when I am trying to upload to Google Docs by API.

We are using the Java library, and we are using DEFAULT_CHUNKED_SIZE, but it failed to upload any files.
Then I changed to NO_CHUNKED_MEDIA_REQUEST, we were able to upload small files , however large file size still fail.

I also tried to set a specific value, but that also failed.

reason:Error writing request body to server

service = new DocsService(applicationName);
service.setChunkedMediaUpload(512*1024);

Strange enough, this only happens on one of our hosting environment. Another instance which runs on Amazon EC2 doesn't
have this issue at all.

Does this has something to do with server configuration, JVM version etc.?

 

Thx

Jerry

[Jerry]

BTW, the exception log is:

 

java.io.IOException: Error writing request body to server
 at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.checkError(HttpURLConnection.java:2637)
 at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(HttpURLConnection.java:2620)
 at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
 at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
 at com.google.gdata.data.media.MediaSource$Output.writeTo(MediaSource.java:90)
 at com.google.gdata.data.media.MediaBodyPart$MediaSourceDataHandler.writeTo(MediaBodyPart.java:74)
 at javax.mail.internet.MimeBodyPart.writeTo(MimeBodyPart.java:1350)
 at javax.mail.internet.MimeBodyPart.writeTo(MimeBodyPart.java:845)
 at javax.mail.internet.MimeMultipart.writeTo(MimeMultipart.java:361)
 at com.google.gdata.wireformats.output.media.MediaMultipartGenerator.generate(MediaMultipartGenerator.java:58)
 at com.google.gdata.wireformats.output.media.MediaMultipartGenerator.generate(MediaMultipartGenerator.java:37)
 at com.google.gdata.client.Service.writeRequestData(Service.java:1813)
 at com.google.gdata.client.media.MediaService.insert(MediaService.java:389)
 at com.ricoh.mfpsaas.googledocs.DocumentList.uploadFile(Unknown Source)
 at com.ricoh.mfpsaas.googledocs.GoogleAPI.callGoogleDocsUploadFile(Unknown Source)
 at com.ricoh.mfpsaas.scheduler.helpers.CloudTaskGoogleDocsHelper.sendOriginalFileToGoogleDocs(Unknown Source)
 at com.ricoh.mfpsaas.scheduler.CloudTaskJob.doStorageJob(Unknown Source)
 at com.ricoh.mfpsaas.scheduler.CloudTaskJob.execute(Unknown Source)
 at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
 at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:534)

[Vic Fryzel]

I'm on a plane, so am unable to setup a test environment for this.

However, try a chunk size of 512 bytes.  If that doesn't work, try a chunk size of < 512 bytes (e.g. 256 bytes.)

Thanks,

-Vic

[hitoshi uchida]

Hi Jerry,
Your http version seems to be HTTP/1.0 which doesn't support Transfer-
Encoding: chunked.
If you can upgrade your environment to HTTP/1.1, you could upload a
large size file.

On 2月17日, 午前9:24, Jerry <jerryjh...@gmail.com> wrote:
> BTW, the exception log is:
>
> java.io.IOException: Error writing request body to server
>  at

> sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.checkError(...)
>  at
> sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(HttpU...)

>  at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
>  at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
>  at
> com.google.gdata.data.media.MediaSource$Output.writeTo(MediaSource.java:90)
>  at

> com.google.gdata.data.media.MediaBodyPart$MediaSourceDataHandler.writeTo(Me diaBodyPart.java:74)

>  at javax.mail.internet.MimeBodyPart.writeTo(MimeBodyPart.java:1350)
>  at javax.mail.internet.MimeBodyPart.writeTo(MimeBodyPart.java:845)
>  at javax.mail.internet.MimeMultipart.writeTo(MimeMultipart.java:361)
>  at
> com.google.gdata.wireformats.output.media.MediaMultipartGenerator.generate( MediaMultipartGenerator.java:58)
>  at

> com.google.gdata.wireformats.output.media.MediaMultipartGenerator.generate( MediaMultipartGenerator.java:37)

[Jerry]

Vic,

 

Setting to 512/256 bytes makes no difference, it still give the same exception ...

 

 

[Jerry]

Uchida-san,

 

Looks like we are using HTTP 1.1, my colleague sent you an Email with the packet capture, please help.

 

Thx

Jerry

[Vic Fryzel]

Did you attempt what I mentioned in the previous message?

-Vic

[Vic Fryzel]

Can you paste your current code to me?  You're not still doing the *1024 are you?

-Vic

[Hitoshi Uchida]

Hi Jerry,
Can you check the failure environment has HTTP/1.0 proxies or not before reaching the google docs server ?

--
Hitoshi Uchida

[hitoshi uchida]

Hi Jerry,

Can you check the failure environment has HTTP/1.0 proxies or not
before reaching the google docs server ?

If HTTP/1.0 proxies exist, you would fail to upload a large size
document via browser also.
Though I can't take a look the packets now, I will do later.

[Hitoshi Uchida]

Hi Jerry,

Sorry for my late response.
Though you tried to send a scanned pdf using multipart in Time
52.396330 of the captured packets, the http server send TCP RST in
Time 52.396605 immediately. The reason of RST from the server seems to
be 0 of windows size.

2011/2/23 hitoshi uchida <hitoshi...@gmail.com>:

--
Regards,
Hitoshi Uchida <hitoshi...@gmail.com>

[Jerry]

Uchida-san,

 

We do have a HTTP 1.0 firewall.

Is there anyways to programatically force the GoogleDocs API to use HTTP protocol version 1.0 instead of 1.1, and turn off the chunck encoding.

 

Thx

Jerry

[Vic Fryzel]

I'm unaware of a way to do this, unfortunately.  I'll ask around just in case.

-Vic

[Hitoshi Uchida]

Hi Jerry,

Do you connect the google docs server with SSL/TLS ? I think you are connecting with no-encrypted because the wirewall could block.
So if you change to SSL/TLS, your request could be reached because the wirewall couldn't check the transfered message.

--
Hitoshi Uchida

[Jerry]

Uchida-san,

 

 

In our DocumentList.java, we do have

 

public static final String DEFAULT_AUTH_PROTOCOL = "https";
public static final String DEFAULT_AUTH_HOST = "docs.google.com";

public static final String DEFAULT_PROTOCOL = "https";
public static final String DEFAULT_HOST = "docs.google.com";

 

Thx

Jerry

[Jerry]

Vic,

 

Really appreciate that!

 

Thx

Jerry