New chunked upload mechanism supports pause and resume during uploads

[Greg Robbins]

The top-of-trunk library now uses the new chunked, resumable upload protocol for uploading to Google Docs and to YouTube.

The new upload protocol allows files to be sent to the server in chunks rather than all at once; this may improve reliability when uploading large files.

In addition, the new protocol allows uploads to be paused, so tickets now support the methods -pauseUpload, -resumeUpload, and -isUploadPaused. The sample Docs and YouTube applications demonstrate pause and resume for uploads.


NOTE: While the new upload protocol is mostly transparent to existing apps, if your application uploads documents to Google Docs, your code may need to be changed slightly. In particular, to upload a document to the user's feed, instead of inserting to the feed's postLink, you will now insert to the feed's uploadLink, like this:

NSURL *uploadURL = [[docListFeed uploadLink] URL];
ticket = [service fetchEntryByInsertingEntry:newEntry
  forFeedURL:uploadURL
  delegate:self
  didFinishSelector:@selector(uploadFileTicket:finishedWithEntry:error:)];

For replacing (updating) uploaded files, entries now have an uploadEditLink to use instead of the editLink.


For better reliability when uploading, remember to tell the service object to automatically retry failed fetches:
  [service setIsServiceRetryEnabled:YES];


The library's default chunk size for uploads varies depending on the operating system version, but you can optionally specify it for your service object if you want to force large or small upload chunks, like:
  [self setServiceUploadChunkSize:1000000];


When uploading large files, consider using NSData's +dataWithContentsMappedFile: rather than +dataWithContentsOfFile: to avoid using memory to hold the entire file.


If you are interested in the underlying upload protocol, it is described at
  http://code.google.com/apis/documents/docs/3.0/developers_guide_protocol.html#ResumableUpload

It is instructive to turn on http logging to see just what goes on "behind the scenes" during file uploads.

[Greg Robbins]

In other words...

If your application uploads files to Google Docs or YouTube, you should rebuild and test against the top-of-trunk library sources. If there is a compatibility issue, the time to find out is now, not in a few weeks when the next release of the library is out.

YouTube apps should not require any changes. Docs apps may need to change the URL they use when inserting new files from the feed's postLink to its uploadLink.

[JustinXXVII]

How long before postLink no longer works?

[Greg Robbins]

The feed's upload link is for chunked uploads; the feed's post link is for non-chunked (multipart MIME) uploads. The library will continue to support both, at least as long as any of the GData services still require non-chunked uploads. 

In the top-of-trunk library sources, Docs and YouTube now default to a non-zero upload chunk size. When the upload chunk size is non-zero, the insert must be done to the feed's upload link, not to the post link.

After allocating the service object, you could explicitly set the service's upload chunk size to zero. Then inserting would need to be done to the post link, not to the upload link. 

I don't know if Docs or YouTube APIs will stop accepting non-chunked uploads in the near future.

[Charbax]

Are there some Docs/Youtube uploader apps for Windows or Ubuntu that
allows to automatically resume uploads to Youtube and Google Docs?

I have an unstable upload connection that keeps getting reset by the
ISP, I need some client uploader software solution to auto-resume
uploads to these services.

[Frank Mantek]

Yes, there are. Look here:

http://groups.google.com/group/gdata-dotnet-client-library/browse_thread/thread/86fa48d57817b320

Frank Mantek
Google

[wally76]

hallo Frank

I understood the resumable upload feature which looks quite new, and
the youtube uploader as well.

Any chance to see some day also a Docs uploader?

I am a user (not a developer, at least in this case) who would like to
use Docs in order to store large backup files. A nice and quite
powerful uploader client would be very appreciated, then.

thanks for your feedback

br
Valerio

On Feb 22, 12:23 pm, Frank Mantek <fman...@gmail.com> wrote:
> Yes, there are. Look here:
>

> http://groups.google.com/group/gdata-dotnet-client-library/browse_thr...

>
> Frank Mantek
> Google
>
> On Feb 20, 7:13 pm, Charbax <char...@gmail.com> wrote:
>
>
>
> > Are there some Docs/Youtube uploader apps for Windows or Ubuntu that
> > allows to automatically resume uploads to Youtube and Google Docs?
>
> > I have an unstable upload connection that keeps getting reset by the
> > ISP, I need some client uploader software solution to auto-resume

> > uploads to these services.- Hide quoted text -
>
> - Show quoted text -