Google Cloud Storage, appengine/file, storage and Go Status Quo?

[Ralf Rottmann]

At the risk of being a pain: Can anybody from Google (Glenn perhaps?) give an update on the current status of Google Cloud Storage support for Go on App Engine?

For some reason I cannot find the code for the storage package anymore here [1]. The page no longer exists.

The appengine/file package has been deprecated and is thus no longer supported on Go for Managed VMs. With the storage package no longer existing and the file package gone, how are we supposed to use GCS?

The sample application here [2] makes use of the storage package - which does not exist anymore.

Having a reliable solution for GCS is super important to us. The somewhat unpredictable state makes us feel pretty uncomfortable. Why is the storage package gone?

[1] https://code.google.com/p/google-api-go-client/storage/v1beta2
[2] https://github.com/GoogleCloudPlatform/storage-getting-started-go/blob/master/storage-sample.go
--
--
rottmann.net | Google+ <https://plus.google.com/106658337982640166595> |
Twitter <http://twitter.com/ralf> | Facebook<http://www.facebook.com/ralfrottmann>|
LinkedIn <http://de.linkedin.com/in/rottmann>

[Andrew Gerrand]

The storage package isn't gone; that's just a bug in code.google.com not showing you the code.

You can view the docs here:

  http://godoc.org/code.google.com/p/google-api-go-client/storage/v1beta2

This recipe should get you started:

  http://play.golang.org/p/dmYteB8hZW

I apologize that I don't have time to write up something more complete; I'm about to get on a flight.

The missing piece of the puzzle is the goauth2/appengine/serviceaccount package. We're working on making this better integrated. Please note that Managed VMs are still in the early testing stage; this will be a lot more accessible and well documented for launch.

Thanks for your patience.

Andrew

[Ralf Rottmann]

Thanks Andrew,

Brilliant as always. The only part that's missing is how to actually store a file / blob then. I'll try to port the appengine/file based code we've got later tonight. If everything goes fine, I'll provide the complete sample.

If anybody else already got it working, feel free to copy in here!

[Ian Rose]

We are using GCS pretty heavily from both App Engine and GCS and it works well enough although there are a few rough spots (e.g. this issue).

If you run into specific problems, feel free to reply here and hopefully I might have run into something similar...

- Ian

[Glenn Lewis]

Did my comment not resolve the issue for you?

If not, can you please further describe the problems that you are having within that issue report?

Thanks!

-- Glenn

--
You received this message because you are subscribed to the Google Groups "google-appengine-go" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengin...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ian Rose]

Sorry about that - just replied on the ticket!

[Prateek Malhotra]

Sorry to hijack the thread, but this open issue addresses a caching issue in the goauth2/appengine/serviceaccount package: https://code.google.com/p/goauth2/issues/detail?id=28

 It hasn't been acknowledged yet and I was wondering how active the issue tracker on that package really was.

Thank you,

Prateek

[Machiel Groeneveld]

The current status is still a bit troubling, Java/Python have new GCS clients and are well documenten. Golang has to get by with some broken links and a deprication (while the blob API is still needed to serve files directly). It would be awesome if the Go support can be pulled up to real production level and/or a roadmap would be available.

Op woensdag 23 april 2014 22:54:19 UTC+2 schreef Ralf Rottmann:

[Ian Rose]

IMO the google-api-go-client isn't all that bad to use with the lone exception of the goauth2 serviceaccounts bug that Prateek mentioned above.  If you access GCS with any degree of frequency, you can very easily exceed your quota of access tokens due to buggy caching.  Once that gets fixed (soon I hope!  especially since a proposed patch has already been submitted), I feel like GCS support will be in pretty decent shape.  Sure the API (since it is machine generated) won't be quite as nice as a hand-written client like in python/java, but I, at least, definitely have higher priorities than that!

[Glenn Lewis]

Regarding the goauth2 serviceaccounts bug, I am working on fixing that.

If anyone has a simple test app that demonstrates the problem before/after the patch, that would help tremendously.

Thanks!

-- Glenn

--

[Prateek Malhotra]

I came across this while testing the BigQuery API for streaming click stats to a table. You'll also have to fix an outstanding bug with the BigQuery Go Client (https://code.google.com/p/google-api-go-client/issues/detail?id=15#c5).

You can write a simple client to rapidly make requests against the handler that's streaming data to BigQuery. Try it with and without the patch.

AppEngine code (untested):

http://play.golang.org/p/pkzzCvHF5O

Now you just need a Go client or some sort of load tester to hit the handler as fast as your machine hundreds of times a second

[Prateek Malhotra]

Now that I think about it, I think one of the errors I ignored is required: http://play.golang.org/p/Z-4c4BPkWQ