Flexble environment returns 404 for servlet requests, the same app runs fine in standards environment

[Arnold Minde]

Hi,

I have used flexible runtime for a backend for weeks on this app, until after a resent update the backend started to give 404 on everything except static files. Now it appears that any deployment of flexible environment module on the app does not execute servlets.

To show this, I have uploaded a plain 1 servlet + 1 static file app on two new modules, 

1) module test on standard environment 

2) module testvm on flexible environment. 

Both modules return static content without a problem:

https://test-dot-gazeting-test.appspot.com/

https://testvm-dot-gazeting-test.appspot.com/

However, 404 is returned by the flexible environment module when calling a servlet url

https://test-dot-gazeting-test.appspot.com/example

https://testvm-dot-gazeting-test.appspot.com/example

The servlet mapped to /example only contains:

@Override

protected void doGet(HttpServletRequest req, HttpServletResponse resp)

throws ServletException, IOException {

resp.getWriter().println("ExampleServlet is good");

}

The only difference between the two modules are the entries <module> and <vm> in the appengine-web.xml. I have tried to disable pre-compilation, but it did not make a difference.

This only seems to affect new deployment, because I have an older version of another module that works just fine, any new version for it fails.

Could anyone please share a clue about what might be the problem? Or what I might be missing?

Thank you.

[Ben Rometsch]

I have exactly the same problem. Need an urgent solution! I have 2 flexible VMs running one app. Both servers just got restarted as part of their weekly upgrade and are now completely broken! Is anyone at Google looking at this? 

[Ben Rometsch]

Im seeing this on the STDOUT logs when I ssh into the machine - not sure how relevant it is? 

2016-09-02 16:43:46.730:WARN:oejx.XmlConfiguration:main: Config error at <Call name="init"><Arg><SystemProperty name="jetty.home" default="."/>/webapps/root</Arg><Arg><SystemPro

perty name="appengine_web_xml" default="WEB-INF/appengine-web.xml"/></Arg></Call> java.lang.reflect.InvocationTargetException in file:/home/vmagent/appengine-java-vmruntime/cont

exts/root.xml

2016-09-02 16:43:46.731:WARN:oejd.DeploymentManager:main: Unable to reach node goal: started

java.lang.reflect.InvocationTargetException

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.eclipse.jetty.util.TypeUtil.call(TypeUtil.java:500)

        at org.eclipse.jetty.xml.XmlConfiguration$JettyXmlConfiguration.call(XmlConfiguration.java:730)

        at org.eclipse.jetty.xml.XmlConfiguration$JettyXmlConfiguration.configure(XmlConfiguration.java:417)

        at org.eclipse.jetty.xml.XmlConfiguration$JettyXmlConfiguration.configure(XmlConfiguration.java:354)

        at org.eclipse.jetty.xml.XmlConfiguration.configure(XmlConfiguration.java:262)

        at org.eclipse.jetty.deploy.providers.WebAppProvider.createContextHandler(WebAppProvider.java:287)

        at org.eclipse.jetty.deploy.App.getContextHandler(App.java:101)

        at org.eclipse.jetty.deploy.bindings.StandardDeployer.processBinding(StandardDeployer.java:36)

        at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:186)

        at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:498)

        at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:146)

        at org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:180)

        at org.eclipse.jetty.deploy.providers.WebAppProvider.fileAdded(WebAppProvider.java:461)

        at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:64)

        at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:609)

        at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:528)

        at org.eclipse.jetty.util.Scanner.scan(Scanner.java:391)

        at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:313)

        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)

        at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:150)

        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)

        at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:560)

        at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:235)

        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)

        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)

        at org.eclipse.jetty.server.Server.start(Server.java:387)

        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)

        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)

        at org.eclipse.jetty.server.Server.doStart(Server.java:354)

        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)

        at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1255)

        at java.security.AccessController.doPrivileged(Native Method)

        at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1174)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.eclipse.jetty.start.Main.invokeMain(Main.java:321)

        at org.eclipse.jetty.start.Main.start(Main.java:817)

        at org.eclipse.jetty.start.Main.main(Main.java:112)

Caused by: 

java.lang.UnsupportedClassVersionError: com/google/appengine/repackaged/com/google/common/base/MoreObjects : Unsupported major.minor version 52.0

        at java.lang.ClassLoader.defineClass1(Native Method)

        at java.lang.ClassLoader.defineClass(ClassLoader.java:803)

        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)

        at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)

        at java.net.URLClassLoader.access$100(URLClassLoader.java:71)

        at java.net.URLClassLoader$1.run(URLClassLoader.java:361)

        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

[Arnold Minde]

Yes Ben. Thanks for the SSH tip. I can confirm I see the same thing on the logs:

Caused by: java.lang.UnsupportedClassVersionError: com/google/appengine/repackaged/com/google/common/base/MoreObjects : Unsupported major.minor version 52.0 at java.lang.ClassLoader.defineClass1(Native Method)

I have a version of a module that I deployed on the 1st of September that works fine. Any new deployment from the 2nd fails. Feels like something change on Google's side yesterday.

[Arnold Minde]

Ben, using the latest version of SDK 1.9.42 seem to have fixed it for me. May be try that. I'll post an update here if it is a fluke.

On Friday, September 2, 2016 at 8:48:37 PM UTC+3, Ben Rometsch wrote:

[Ben Rometsch]

yes that worked! Thanks! 

That's CRAZY...

[Arnold Minde]

Glad to hear that.