Is Google Cloud Storage Java API broken now?

[Tapir]

Now, on my website, new designs saved in Google Cloud Storage are all loading failed, but the old designs saved in bigtable are still ok.

http://www.phyard.com/profile/Tapir/designs

But the document https://developers.google.com/appengine/docs/java/googlestorage/ doesn't confirm this.

And Google never said this API has been removed.

[Tapir]

yesterday, all designs are loading ok, including the one saved in Google Cloud Storage. 

[Vinny P]

On Thu, Oct 31, 2013 at 2:21 PM, Tapir <tapi...@gmail.com> wrote:

yesterday, all designs are loading ok, including the one saved in Google Cloud Storage. 

Can you look into the Cloud Storage bucket and confirm that the files are still there, and the bucket/files have the correct permissions configured?

To do this, go to the Google Cloud Console, click the cloud project being used, and click the Cloud Storage service. Then open up the bucket that stores your files.

 

 

-----------------

-Vinny P

Technology & Media Advisor

Chicago, IL

App Engine Code Samples: http://www.learntogoogleit.com

[Tapir]

On Friday, November 1, 2013 9:01:15 AM UTC+8, Vinny P wrote:

On Thu, Oct 31, 2013 at 2:21 PM, Tapir <tapi...@gmail.com> wrote:

yesterday, all designs are loading ok, including the one saved in Google Cloud Storage. 

Can you look into the Cloud Storage bucket and confirm that the files are still there, and the bucket/files have the correct permissions configured?

To do this, go to the Google Cloud Console, click the cloud project being used, and click the Cloud Storage service. Then open up the bucket that stores your files.

There is no "Cloud Storage service" listed in the Google Cloud Console for my this project. This project was created before the Google Cloud Console was released.

All my projects created before the Google Cloud Console was released have no "Cloud Storage service" listed in the Google Cloud Console.

Before the Google Cloud Console was released, I check my buckets from another url entry, but now I can't find it in google's documents.

The only entry I can find in the documents is Google Cloud Console.

I have done nothing on my this project for a week, and I have done nothing on the "Cloud Storage service" part of this project for months.

Did google remove all my data?!

[Tapir]

Ok, I just checked my "Cloud Storage service" page for my this project by copyimg and modifying the url "#/project/apps~another-project/storage" from my another newly created project to "#/project/apps~this-project/storage".

Then what?! A tip with yellow background says "Please enable billing to use all features.Dismiss".

But this project is really billing enabled, when I clicked "Billing" item for this project, it says "Billing for this project is managed from the App Engine Admin Console."

I really enabled billing in the app engine console.

So it seems google separate the billing into two parts. One part for app engine, one part for cloud storage.

To googlers, is it ok I enabled billing for the cloud storage part safely (without losing my any data) now?

[Vinny P]

On Thu, Oct 31, 2013 at 11:58 PM, Tapir <tapi...@gmail.com> wrote:

There is no "Cloud Storage service" listed in the Google Cloud Console for my this project. This project was created before the Google Cloud Console was released.

All my projects created before the Google Cloud Console was released have no "Cloud Storage service" listed in the Google Cloud Console.

Before the Google Cloud Console was released, I check my buckets from another url entry, but now I can't find it in google's documents.

The only entry I can find in the documents is Google Cloud Console.

You can also look into a Cloud Storage bucket using gsutil: https://developers.google.com/storage/docs/gsutil 

[Tapir]

On Friday, November 1, 2013 1:56:03 PM UTC+8, Vinny P wrote:

On Thu, Oct 31, 2013 at 11:58 PM, Tapir <tapi...@gmail.com> wrote:

There is no "Cloud Storage service" listed in the Google Cloud Console for my this project. This project was created before the Google Cloud Console was released.

All my projects created before the Google Cloud Console was released have no "Cloud Storage service" listed in the Google Cloud Console.

Before the Google Cloud Console was released, I check my buckets from another url entry, but now I can't find it in google's documents.

The only entry I can find in the documents is Google Cloud Console.

You can also look into a Cloud Storage bucket using gsutil: https://developers.google.com/storage/docs/gsutil 

I bless the bucket and all objects in it are still there.

[Amy Unruh]

Tapir,

From your description, I am guessing that your app engine app was created a while ago, and doesn't have an integrated cloud console project automatically associated with it.  

This means that when you set up your use of Cloud Storage (GCS) for your app, you likely would have created a separate Cloud project, and then added your app engine's service account to the 'Teams' panel for that project, so that your app engine app could access the GCS bucket.

So, the first step is to reconstruct which cloud project, and which GCS bucket in that project, your app is trying to connect to.

Then, once you figure out which project and GCS bucket you are using, Vinny's suggestions for getting more info about the bucket contents are good ones.  A good first step is to see if the data is still there as you expect.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[Tapir]

On Friday, November 1, 2013 1:56:03 PM UTC+8, Vinny P wrote:

On Thu, Oct 31, 2013 at 11:58 PM, Tapir <tapi...@gmail.com> wrote:

There is no "Cloud Storage service" listed in the Google Cloud Console for my this project. This project was created before the Google Cloud Console was released.

All my projects created before the Google Cloud Console was released have no "Cloud Storage service" listed in the Google Cloud Console.

Before the Google Cloud Console was released, I check my buckets from another url entry, but now I can't find it in google's documents.

The only entry I can find in the documents is Google Cloud Console.

You can also look into a Cloud Storage bucket using gsutil: https://developers.google.com/storage/docs/gsutil 

$ gsutil ls
GSResponseError: status=403, code=AccessDenied, reason="Forbidden", message="Access denied.", detail="this-project-id is not a valid project id spec"

Hope it is caused for not billing.

[Tapir]

On Friday, November 1, 2013 2:19:19 PM UTC+8, Amy Unruh wrote:

Tapir,

From your description, I am guessing that your app engine app was created a while ago, and doesn't have an integrated cloud console project automatically associated with it.  

This means that when you set up your use of Cloud Storage (GCS) for your app, you likely would have created a separate Cloud project, and then added your app engine's service account to the 'Teams' panel for that project, so that your app engine app could access the GCS bucket.

So, the first step is to reconstruct which cloud project, and which GCS bucket in that project, your app is trying to connect to.

Yes, you are right!

I connected the Google Cloud Storage with my app engine app in the API Console: https://code.google.com/apis/console/, about more than one year ago.

Ok, I remember yesterday I really deleted two newly projects I created after Google Compute Engine is released. I promise the two projects have nothing relation with the Google Cloud Storage service.

Would google remove my data occasionally with the removal of the two project? I'm some nervous now.

[Tapir]

It is very strange! When I click the "Team" link in API console, the browser will open the "Permission" page of my another newly created Compute Engine project, not the old app engine project I ever connected with the Cloud Storage service.

[Tapir]

BTW, on the "Billing" page of the API console, it says "Billing is enabled for all active".

 

So I'm sure I have ever enabled billing for the Google Cloud Storage service.

[Amy Unruh]

Tapir,

I think the best way to diagnose is to use the web interface: start from your cloud console (cloud.google.com/console) and look at each of your existing projects in turn. Find the one that, in its Cloud Storage (GCS) panel, lists the bucket that you're using for your app.  See what's in that bucket.

(If you have recently deleted some projects, and also your app has recently started to have GCS issues, then it does sound like you might have accidentally deleted the project that included the bucket you were using.  But, go through each of your existing projects and check their GCS buckets, and that will give you more definitive info).

[Tapir]

When I click the "Google Cloud Storage Manager" link in the "Google Cloud Storage" page of the API Console, it jumps to another newly created Compute Engine project, https://cloud.google.com/console#/project/apps~another-project-id/storage, not the old app engine project I ever connected with the Cloud Storage service.Show trimmed content (same as above I descripted for "Team" link). Surely, there is nothing stored in the newly created Compute Engine project.

When I manually change the url to https://cloud.google.com/console#/project/apps~this-project-id/storage, it says "Please enable billing to use all features.Dismiss" in a text box with yellow background.

What, I'm really confused! It is really confessing!

.

[Tapir]

On Friday, November 1, 2013 2:59:00 PM UTC+8, Amy Unruh wrote:

Tapir,

I think the best way to diagnose is to use the web interface: start from your cloud console (cloud.google.com/console) and look at each of your existing projects in turn. Find the one that, in its Cloud Storage (GCS) panel, lists the bucket that you're using for your app.  See what's in that bucket.

As I said above, there is no "Cloud Stoage" page for all my old app engine projects. I must replace copy the url for a newly created compute engine projects and replace the id part in the url to the old app engine project id.

 

(If you have recently deleted some projects, and also your app has recently started to have GCS issues, then it does sound like you might have accidentally deleted the project that included the bucket you were using.  But, go through each of your existing projects and check their GCS buckets, and that will give you more definitive info).

The deleted two project are all Compute Engine projects I created recently. I kust used them as VPNs. I have never connected them with my old Cloud Storage service.

[Tapir]

One of the two deleted Compute Engine is the first my ever created Compute Engine project.

After the removals, the Compute Engine project I mentioned above with project "another-project-id" becomes the exiting oldest created Compute Engine project.

Will Compute Engine team connect my old existed Cloud Storage service to the oldest created Compute Engine project automatically?

[Tapir]

Is there a way to cancel removing my Compute Engine projects? 

[Tapir]

On Friday, November 1, 2013 2:59:00 PM UTC+8, Amy Unruh wrote:

Tapir,

I think the best way to diagnose is to use the web interface: start from your cloud console (cloud.google.com/console) and look at each of your existing projects in turn. Find the one that, in its Cloud Storage (GCS) panel, lists the bucket that you're using for your app.  See what's in that bucket.

(If you have recently deleted some projects, and also your app has recently started to have GCS issues, then it does sound like you might have accidentally deleted the project that included the bucket you were using.  But, go through each of your existing projects and check their GCS buckets, and that will give you more definitive info).

Ok, I just went through all my projects. All my compute engine project's Cloud Storage pages are list nothing, no buckets at all. All my old app engine proejcts with master-lave datastore show "Error The project you requested is unavailable." All my old app engine with high-replication datastore shows "Please enable billing to use all features.Dismiss".

[Amy Unruh]

Tapir,

The cloud console lists not only your app engine apps, but all your Cloud projects, including projects where you'd enabled Compute Engine or other cloud services (e.g. Cloud Storage, BigQuery, Cloud SQL, etc.).

So that is why I suggested you just go systematically through that listing and see if you can find your bucket. (I understand that the appengine-only entries will not have Cloud Storage enabled).

A Cloud project can have multiple services enabled.  So, you could have enabled Cloud Storage in the *same* project where you'd enabled Compute Engine. (And, if you delete such a project, you're deleting not just your compute engine setup, but your setup for all the services you'd enabled in that project).

Because you'd set things up a while ago, before app engine apps became integrated with cloud projects, I suspect that you set up a Cloud Storage bucket in one of your other cloud projects, then gave your app engine app permission to connect with it by adding its service account to the 'team' information in the project.  From the error you're getting, I suspect this may have been one of your Compute Engine-enabled projects, that you've now deleted.

[Tapir]

On Friday, November 1, 2013 5:14:50 PM UTC+8, Amy Unruh wrote:

Tapir,

The cloud console lists not only your app engine apps, but all your Cloud projects, including projects where you'd enabled Compute Engine or other cloud services (e.g. Cloud Storage, BigQuery, Cloud SQL, etc.).

So that is why I suggested you just go systematically through that listing and see if you can find your bucket. (I understand that the appengine-only entries will not have Cloud Storage enabled).

A Cloud project can have multiple services enabled.  So, you could have enabled Cloud Storage in the *same* project where you'd enabled Compute Engine. (And, if you delete such a project, you're deleting not just your compute engine setup, but your setup for all the services you'd enabled in that project).

Because you'd set things up a while ago, before app engine apps became integrated with cloud projects, I suspect that you set up a Cloud Storage bucket in one of your other cloud projects, then gave your app engine app permission to connect with it by adding its service account to the 'team' information in the project.  From the error you're getting, I suspect this may have been one of your Compute Engine-enabled projects, that you've now deleted.

I can confirm:

1. I have never connected Cloud Storage with any compute engine type projects. I only ever connected it to one my app engine style project. (100% for sure)

2. before yesterday, the app engine project run well, which means the Cloud Storage connects with the app engine project for sure.

[Tapir]

On Friday, November 1, 2013 5:14:50 PM UTC+8, Amy Unruh wrote:

Tapir,

The cloud console lists not only your app engine apps, but all your Cloud projects, including projects where you'd enabled Compute Engine or other cloud services (e.g. Cloud Storage, BigQuery, Cloud SQL, etc.).

So that is why I suggested you just go systematically through that listing and see if you can find your bucket. (I understand that the appengine-only entries will not have Cloud Storage enabled).

A Cloud project can have multiple services enabled.  So, you could have enabled Cloud Storage in the *same* project where you'd enabled Compute Engine. (And, if you delete such a project, you're deleting not just your compute engine setup, but your setup for all the services you'd enabled in that project).

Because you'd set things up a while ago, before app engine apps became integrated with cloud projects, I suspect that you set up a Cloud Storage bucket in one of your other cloud projects, then gave your app engine app permission to connect with it by adding its service account to the 'team' information in the project.  From the error you're getting, I suspect this may have been one of your Compute Engine-enabled projects, that you've now deleted.

And I only did the connecting Cloude Storage and project job once, long before the Compute Engine came out.

[Tapir]

It seems my bucket custom domain designfiles.phyard.com is still ping-able. 

It is just I can't read it.

Hope the datas are not lost and google can help me find them back.

[Brian Dorsey]

On Friday, November 1, 2013 2:49:13 AM UTC-7, Tapir wrote:

On Friday, November 1, 2013 5:14:50 PM UTC+8, Amy Unruh wrote:

Tapir,

The cloud console lists not only your app engine apps, but all your Cloud projects, including projects where you'd enabled Compute Engine or other cloud services (e.g. Cloud Storage, BigQuery, Cloud SQL, etc.).

So that is why I suggested you just go systematically through that listing and see if you can find your bucket. (I understand that the appengine-only entries will not have Cloud Storage enabled).

A Cloud project can have multiple services enabled.  So, you could have enabled Cloud Storage in the *same* project where you'd enabled Compute Engine. (And, if you delete such a project, you're deleting not just your compute engine setup, but your setup for all the services you'd enabled in that project).

Because you'd set things up a while ago, before app engine apps became integrated with cloud projects, I suspect that you set up a Cloud Storage bucket in one of your other cloud projects, then gave your app engine app permission to connect with it by adding its service account to the 'team' information in the project.  From the error you're getting, I suspect this may have been one of your Compute Engine-enabled projects, that you've now deleted.

I can confirm:

1. I have never connected Cloud Storage with any compute engine type projects. I only ever connected it to one my app engine style project. (100% for sure)

2. before yesterday, the app engine project run well, which means the Cloud Storage connects with the app engine project for sure.

Tapir, 

I suspect that Amy's analysis is correct here. In regards to #1, there isn't a manual step to connect Cloud Storage with Compute Engine. App Engine projects are separate from the rest of the APIs for historical reasons, but all other cloud APIs can be activated on a single project. 

Since this worked recently, and you also recently deleted a project with Compute Engine enabled, it is very likely that this project also had your active Cloud Storage buckets as well.

Please visit: https://cloud.google.com/console#/project

This will give you a list of all of the projects you have access to. 

Are any of them labeled "pending deletion" in red? If so, please check the box next to that project and click the "Undo Deletion" button. 

If none are labeled "pending deletion", please follow Amy's advice from above and check each project to find out which project has your "designfiles.phyard.com" bucket. You can do that by clicking on a project name, clicking "Cloud Storage" (from the left-side list) and looking at the list of buckets displayed. 

Once you find the project with your bucket, you may need to re-enable billing. To do that, click "Billing" (from the left-side list), and click "Enable Billing".

If this doesn't work for you, please email gs-...@google.com and I will help you via our support system.

Take care,

-Brian

[Tapir]

Hi Brian,

thanks for your guilds.

No "pending deletion" projects listed now. When I removed a compute engine project, it disappeared right now, even the app engine app connected with the compute engine project are also disappeared right now.

And I can't find my buckets now, in any project. I have already enabled billing for the Cloud Storage last year. The app engine app (phyard.com) connected with Cloud Storage is also billing enabled for a long time.

I have emailed to gs-...@google.com and gc-...@google.com , but not get response yet.

[Tapir]

Ha Brian,

just received your email and notice you can googler.

now, the page https://cloud.google.com/console#/project/apps~youyards/storage shows "Please enable billing to use all features.Dismiss" box with yellow background.

Is it ok if I click the "enable billing" link?

[Tapir]

On Saturday, November 2, 2013 2:16:30 AM UTC+8, Tapir wrote:

Ha Brian,

just received your email and notice you can googler.

sorry for the spelling mistakes. "you are a googler". :)

[Brian Dorsey]

Hello Tapir,

I responded via gs-team. Let's continue this conversation there, so I can help with your specific project ids. 

Take care,

-Brian

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/YxsTLbi55JI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.

[Tapir]

Fortunately my data is restored by cancelling removing my compute engine project.

Thanks Brian!