Upcoming changes to App Engine support in the Google Cloud SDK

[Andrew Jessup]

Hi folks,

We're making an important change to the Google Cloud SDK to simplify how App Engine developers can access and work with the SDK.

As of the next release, 0.9.68, of the Google Cloud SDK - we will no longer be distributing the stand alone App Engine SDKs. 

The gcloud preview app deploy and gcloud preview app run commands that most developers use within gcloud will continue to work, users should not notice any obvious change.

For users who were directly using appcfg.py, appcfg.sh, dev_appserver.py or dev_appserver.sh scripts, or the GUI launchers that were bundled with the Google Cloud SDK, you have two options:

You can continue to use the latest versions of these tools used by installing them from the stand-alone App Engine SDKs that can be found here: https://cloud.google.com/appengine/downloads.

In the short term, if you need to prevent this from happening, you can pin a version of gcloud SDK with the following commands:

  gcloud config set --scope=installation component_manager/fixed_sdk_version 0.9.67 

  gcloud components update

As part of this change, the gae-java, gae-python, gae-php, and gae-go components no longer exist in the component manager.  The `gcloud preview app run` command will install language specific runtime libraries on demand, as needed by your application.

If you have questions or concerns, please let us know.

Thanks,

Andrew, on behalf of the Cloud SDK team

Andrew Jessup | Product Manager, Google Cloud Platform | jes...@google.com 

[Stefano Ciccarelli]

I don't know why you continue to complicate simple things!

The 'gloud preview app deploy' command is absolutely not comparable with 'appcfg.sh update' because simply is missing features!

Where is the '--enable_jar_splitting' flag? And the '--enable_jar_classes' flag?

And what about bugs?

The last time I tried the 'gloud preview app deploy' command I've noticed that the xml element <max-concurrent-requests> in appengine-web.xml is ignored (can be verified *only* with the old console and not with the new).

So, like a flag that changes direction depending on how the wind changes direction, I have to manually install the sdk and I also need to keep it updated.

Thanks to reinvent the wheel all the time!

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CAEF6f2tMjbP%3DeEcBuB5p2ubscWjsR8RHmoPUvnNopxZxydhEBw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Stefano Ciccarelli 
GAE Application Division 
/ Director 
stefano.c...@mmbsoftware.it 

M.M.B. s.r.l. 
via Granarolo, 177/7 - 48018 Faenza (RA) - Italy 
tel. +39.0546.637711 - fax +39.0546.46077 
www.mmbsoftware.it - in...@mmbsoftware.it

Le informazioni contenute in questa comunicazione sono riservate e destinate esclusivamente alla/e persona/e o all'ente sopra indicati. E' vietato ai soggetti diversi dai destinatari qualsiasi uso, copia, diffusione di quanto in esso contenuto sia ai sensi dell'art. 616 c.p., sia ai sensi del DL n. 196/03. Se questa comunicazione Vi e' pervenuta per errore, Vi preghiamo di rispondere a questa e-mail e successivamente cancellarla dal Vostro sistema.

[pdknsk]

As part of this change, the gae-java, gae-python, gae-php, and gae-go components no longer exist in the component manager.  The `gcloud preview app run` command will install language specific runtime libraries on demand, as needed by your application.

It's not quite clear to me if this also applies to the following components.

| Installed     | gcloud app Go Extensions (Linux, x86_64)      | app-engine-go-linux-x86_64 |  26.4 MB |

| Installed     | gcloud app Java Extensions                    | app-engine-java            |  95.2 MB |

| Installed     | gcloud app Python Extensions                  | app-engine-python          |   6.9 MB |

I don't understood why I have to download 120MB of components on every update which I never use.

[Karl MacMillan]

Hey Andrew - a few questions.

On Jul 8, 2015, at 3:16 AM, 'Andrew Jessup' via Google App Engine <google-a...@googlegroups.com> wrote:

Hi folks,

We're making an important change to the Google Cloud SDK to simplify how App Engine developers can access and work with the SDK.

As of the next release, 0.9.68, of the Google Cloud SDK - we will no longer be distributing the stand alone App Engine SDKs. 

Can you give us a timeframe for this release? Many of us probably have some automation around these tools (I certainly do) and it would be nice to be able to plan when we need to make the changes. Especially as this kind of work provides no benefit for me or my users, so it’s not a high priority.

Also - I just had some interaction with the support team over the urlfetch bug and they were discussing which of the next two GAE SDK releases the bug might be fixed in. Does that mean that there are going to be at least two more releases of the stand-alone SDK? Or does this announcement change that?

Basically - I don’t quite understand how the release schedule of what (to me) are two different SDKs interact. Heck - I’m not even quite clear on how these two SDKs are intended to interact at all (more on that below).

The gcloud preview app deploy and gcloud preview app run commands that most developers use within gcloud will continue to work, users should not notice any obvious change.

You say that most users use the gcloud commands, but I find that very hard to believe. All of your docs point to the old tools (e.g., https://cloud.google.com/appengine/docs/python/gettingstartedpython27/uploading and https://cloud.google.com/appengine/docs/java/tools/uploadinganapp). Also your gcloud docs explicitly state that these tools are beta and not covered by “any SLA or deprecation policy and may be subject to backward-incompatible changes” (https://cloud.google.com/sdk/gcloud-app).

Call me a fuddy-duddy, but I tend to stick to what you guys document as the correct way and avoid things that are in beta and you tell me are not yet supported. Needless to say I’m surprised and annoyed for this casual message from you to change all of that without any change to the public documentation.

Also, will these continue to be under the preview command (which I understood to mean were not yet stable)? If these are going to be the only supported means of interacting with GAE it seems surprising that they are going to be in preview still. If they are going to change it would be nice to have one release where the commands are not in preview but the standalone tools are still supported to ease our migration and just in case there are bugs / breaking changes. Otherwise there is no period where I can migrate my automation _and_ use tools that are officially supported.

For users who were directly using appcfg.py, appcfg.sh, dev_appserver.py or dev_appserver.sh scripts, or the GUI launchers that were bundled with the Google Cloud SDK, you have two options:

You can continue to use the latest versions of these tools used by installing them from the stand-alone App Engine SDKs that can be found here: https://cloud.google.com/appengine/downloads.

Just to make certain that I understand - are you saying that we can use the existing releases and there will be no more releases? Your wording is a little confusing to me.

In the short term, if you need to prevent this from happening, you can pin a version of gcloud SDK with the following commands:

  gcloud config set --scope=installation component_manager/fixed_sdk_version 0.9.67 

  gcloud components update

And what happens if I have the standalone SDK installed and don’t do this? Are the standalone tools deleted? Again - I’m a little vague about the relationship between these. When I first started I installed the standalone App Engine SDK but, when I needed google compute, I installed gcloud. Since then I’ve installed updates through gcloud, the GUI Google Updater, and manually when I had to downgrade because of bugs.

So does gcloud manage the existing App Engine SDK or not? How can I tell for other members of my team?

As part of this change, the gae-java, gae-python, gae-php, and gae-go components no longer exist in the component manager.  The `gcloud preview app run` command will install language specific runtime libraries on demand, as needed by your application.

If you have questions or concerns, please let us know.

I’d like to go ahead and ask that you reconsider this plan. It does not seem well thought out. I’d expect that you at least provide a solid description of time frames and deprecation schedule.

Karl

Thanks,

Andrew, on behalf of the Cloud SDK team

Andrew Jessup | Product Manager, Google Cloud Platform | jes...@google.com 

-- 

[Glenn Lewis]

For Go developers specifically, if you have a "v1" (aka "classic") Go App Engine app, you will download and use the stand-alone App Engine Go SDK Andrew pointed to.

This additionally includes the use of the "goapp" tool for running and deploying.

If you have a "v2" (aka "Managed VM") Go app ("vm: true"), you will continue to use the Google Cloud SDK and the "gcloud" tool for running and deploying (examples).

-- Glenn

[Andrew Jessup]

Hi Karl, thanks for reaching out - there seems to be some understandable confusion around this announcement, I've added a few responses inline.

On Wednesday, 8 July 2015 07:42:53 UTC-7, Karl MacMillan wrote:

Hey Andrew - a few questions.

On Jul 8, 2015, at 3:16 AM, 'Andrew Jessup' via Google App Engine <google-a...@googlegroups.com> wrote:

Hi folks,

We're making an important change to the Google Cloud SDK to simplify how App Engine developers can access and work with the SDK.

As of the next release, 0.9.68, of the Google Cloud SDK - we will no longer be distributing the stand alone App Engine SDKs. 

Can you give us a timeframe for this release? Many of us probably have some automation around these tools (I certainly do) and it would be nice to be able to plan when we need to make the changes. Especially as this kind of work provides no benefit for me or my users, so it’s not a high priority.

We expect this release to be available in the next couple of days, although occasionally delays come up.

I absolutely hear you that this is a pain for automation. We're aiming to minimize breaking changes once we remove the 'preview' label from the gcloud app command, which is why we're doing this now. In the interim, the version pinning described below is one way to minimize the effect/surprise changes when using the Google Cloud SDK.

 

Also - I just had some interaction with the support team over the urlfetch bug and they were discussing which of the next two GAE SDK releases the bug might be fixed in. Does that mean that there are going to be at least two more releases of the stand-alone SDK? Or does this announcement change that? 

We're going to keep distributing the stand-alone SDKs (that is the SDKs not shipped in gcloud) for the forseeable future, this announcement doesn't affect them. What this refers to is the copy of appcfg.py/.sh and and dev_appserver.py/.sh that are bundled with the Google Cloud SDK.

 

Basically - I don’t quite understand how the release schedule of what (to me) are two different SDKs interact. Heck - I’m not even quite clear on how these two SDKs are intended to interact at all (more on that below).

The gcloud preview app deploy and gcloud preview app run commands that most developers use within gcloud will continue to work, users should not notice any obvious change.

You say that most users use the gcloud commands, but I find that very hard to believe. All of your docs point to the old tools (e.g., https://cloud.google.com/appengine/docs/python/gettingstartedpython27/uploading and https://cloud.google.com/appengine/docs/java/tools/uploadinganapp). Also your gcloud docs explicitly state that these tools are beta and not covered by “any SLA or deprecation policy and may be subject to backward-incompatible changes” (https://cloud.google.com/sdk/gcloud-app).

Ah, there might be some confusion here. I was referring to the fact that most users of the Google Cloud SDK to interact with GAE are using "gcloud app run/deploy" rather than the appcfg.py/dev_appserver.py commands.

However, you're quite right - most users of App Engine right now are not yet using the Google Cloud SDK at all, but are using the stand-alone App Engine SDKs and are using the appcfg.py/.sh and dev_appserver.py/.sh commands from there. Fortunately these folks aren't affected by this announcement, this only applies to those using appcfg.py/.sh and dev_appserver.py/.sh from the Google Cloud SDK.

Using the stand-alone App Engine SDKs, and the tools therein, is still the recommended path for interacting with App Engine for anyone not using Managed VMs. 

 

Call me a fuddy-duddy, but I tend to stick to what you guys document as the correct way and avoid things that are in beta and you tell me are not yet supported. Needless to say I’m surprised and annoyed for this casual message from you to change all of that without any change to the public documentation. 

Also, will these continue to be under the preview command (which I understood to mean were not yet stable)? If these are going to be the only supported means of interacting with GAE it seems surprising that they are going to be in preview still. If they are going to change it would be nice to have one release where the commands are not in preview but the standalone tools are still supported to ease our migration and just in case there are bugs / breaking changes. Otherwise there is no period where I can migrate my automation _and_ use tools that are officially supported.

We will continue shipping the current (non-beta) tooling using the stand-alone SDKs as we currently do, it's only distribution via gcloud that is affected. We plan to continue shipping these tools in stand-alone SDKs for the forseeable future.

 

For users who were directly using appcfg.py, appcfg.sh, dev_appserver.py or dev_appserver.sh scripts, or the GUI launchers that were bundled with the Google Cloud SDK, you have two options:

You can continue to use the latest versions of these tools used by installing them from the stand-alone App Engine SDKs that can be found here: https://cloud.google.com/appengine/downloads.

Just to make certain that I understand - are you saying that we can use the existing releases and there will be no more releases? Your wording is a little confusing to me.

We will under the stand-alone App Engine SDKs. We won't continue to release these tools in under the Google Cloud SDK. 

 

In the short term, if you need to prevent this from happening, you can pin a version of gcloud SDK with the following commands:

  gcloud config set --scope=installation component_manager/fixed_sdk_version 0.9.67 

  gcloud components update

And what happens if I have the standalone SDK installed and don’t do this? Are the standalone tools deleted?

Tools installed via the stand-alone SDK are unaffected by this. They will continue to work, and we'll keep updating them.

 

Again - I’m a little vague about the relationship between these. When I first started I installed the standalone App Engine SDK but, when I needed google compute, I installed gcloud. Since then I’ve installed updates through gcloud, the GUI Google Updater, and manually when I had to downgrade because of bugs.

So does gcloud manage the existing App Engine SDK or not? How can I tell for other members of my team?

As of this upcoming release - no. The stand alone App Engine SDKs however will continue to provide (and update) the appcfg.py and dev_appserver.py commands, the Google Cloud SDK will continue to provide gcloud preview app and gcloud preview deploy (and hopefully soon we can drop the 'preview' label from these).

 

As part of this change, the gae-java, gae-python, gae-php, and gae-go components no longer exist in the component manager.  The `gcloud preview app run` command will install language specific runtime libraries on demand, as needed by your application.

If you have questions or concerns, please let us know.

I’d like to go ahead and ask that you reconsider this plan. It does not seem well thought out. I’d expect that you at least provide a solid description of time frames and deprecation schedule.  

I'm sorry for the confusion here. Ironically, part of the motivation for changing gcloud now is so that we can provide a more stable (and more quickly released) support for App Engine customers from gcloud down the line.

If this is affecting your team severely you're welcome to reach out to me directly - jes...@google.com, and we'll see if we can help out.

We're also open to feedback on how we can improve notifications on gcloud changes generally. We have a few improvements planned for the gcloud infrastructure (including release track splitting) that will help with this.

 

Karl

Thanks,

Andrew, on behalf of the Cloud SDK team

Andrew Jessup | Product Manager, Google Cloud Platform | jes...@google.com 

-- 
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsub...@googlegroups.com.

[Karl MacMillan]

The gcloud preview app deploy and gcloud preview app run commands that most developers use within gcloud will continue to work, users should not notice any obvious change.

You say that most users use the gcloud commands, but I find that very hard to believe. All of your docs point to the old tools (e.g., https://cloud.google.com/appengine/docs/python/gettingstartedpython27/uploadingand https://cloud.google.com/appengine/docs/java/tools/uploadinganapp). Also your gcloud docs explicitly state that these tools are beta and not covered by “any SLA or deprecation policy and may be subject to backward-incompatible changes” (https://cloud.google.com/sdk/gcloud-app).

Ah, there might be some confusion here. I was referring to the fact that most users of the Google Cloud SDKto interact with GAE are using "gcloud app run/deploy" rather than the appcfg.py/dev_appserver.py commands.

However, you're quite right - most users of App Engine right now are not yet using the Google Cloud SDK at all, but are using the stand-alone App Engine SDKsand are using the appcfg.py/.sh and dev_appserver.py/.sh commands from there. Fortunately these folks aren't affected by this announcement, this only applies to those using appcfg.py/.sh and dev_appserver.py/.sh from the Google Cloud SDK.

Using the stand-alone App Engine SDKs, and the tools therein, is still the recommended path for interacting with App Engine for anyone not using Managed VMs. 

 

Ok - this seems to be the core of the confusion then and the handling of this changes makes a lot more sense to me. Two more questions then:

1. Will everything eventually be moved into gcloud or will gcloud only be the recommended solution for managed VMs? In other words, will the stand-alone SDK eventually be replaced by the Google Cloud SDK?

2. I assume that having both SDKs installed will work fine for those that need a mix of managed VMs and pure App Engine?

Thanks - Karl

[Andrew Jessup]

Hey Karl,

1. The ambition is that we will eventually make gcloud app run/deploy the preferred way to access App Engine for all customers (Managed VM and regular instances), though as has been pointed out here and elsewhere we have some work to do before we get to that point. When we do get to that point, we'll definitely make sure we're giving folks enough time to move over by supporting both SDKs in parallel for a decent period of time.

In the meantime, I would continue to keep using the stand alone SDKs for the moment when working with non-managed VMs if you want the most stable surface area (for use in integration scripts, etc.). Expect the Cloud SDK equivalents to quickly get better.

2. Yes, it should (and in fact this recent change should make it easier). If you run into problems let us know.

--

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.

To post to this group, send email to google-a...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/8B50B7DB-FD6E-46D1-9D4B-5DC9756317ED%40rakkoon.com.

For more options, visit https://groups.google.com/d/optout.

--

[troberti]

What about a git mirror of the App Engine SDK on github.com/GoogleCloudplatform ? In that way, we can conveniently add it is a submodule to our projects and easily update when needed.

[Elissa Tong]

The change is very sudden. I've been using appcfg.py for some time, until this week it stopped working. I suggest we need better documentation & examples. These changes need to be communicated to the customers. It will be very embarrassing when you're doing a presentation with all your execs or team and this command just stopped working.

I finally got my deployment working:

folder_of_myproject $ gcloud preview app deploy app.yaml --set-default

NOTE: app.yaml asks to remove the application_id and version.

You'll get warnings or errors, so your yaml files need to be updated:

WARNING: The [application] field is specified in file [/Users/myname/workspace/github/cp100-appengine-memcache-python/app.yaml].  This field is not used by gcloud and should be removed.

ERROR: The version [1] declared in [/Users/myname/workspace/github/cp100-appengine-memcache-python/app.yaml] does not match the current gcloud version [20150714t161935].

ERROR: (gcloud.preview.app.deploy) Errors occurred while parsing the App Engine app configuration.

[Michael Prentice]

Wow this has really been a bad experience and a change that certainly hasn't simplified my use of gcloud!

Months ago, I finally found a really nice workflow for upgrading the appengine-java-sdk without causing a lot of reconfiguration in IntelliJ IDEA (http://stackoverflow.com/a/28911743/633107).

Now I just went and updated my components in gcloud and this is completely broken! I can't find the appengine-java-sdk anywhere even though it says that I have "Installed     |gcloud app Java Extensions                  | app-engine-java       | 96.2 MB "

After Googling all over the help pages and docs, there is no mention of this change even in the App Engine SDK release notes. This seems to be the only place on the internet that has information about this breaking change.

This change just adds complexity and makes getting started with appengine-java even harder, especially if you do not use Maven (I don't). Now for the third time in the last year, I need to come up with a completely new workflow for updating my GAE java projects. It's really hard to keep advocating on behalf of the GCP when there are so many breaking changes that affect developers and affect the training / presentations that I give (similar to Elissa's experience below).

I hate to be so negative, but the developer experience on the platform has been significantly painful over the last 2 years. Note that I am now also an AWS customer.

Michael Prentice

GDG Space Coast

Google Cloud Developer Challenge 2nd place (US/EU)