Google app engine: 502 Bad Gateway only on POST requests

[Harris Robin Kalash]

So I deployed a nodeJS script on GAE and whenever I hit a POST endpoint I get a 502 Bad Gateway error. The endpoint is a simple service that grabs a screenshot of a page using phantomJS and returns a JSON that contains the base64 representation of the image in it. 

Doing a GET to this endpoint works fine and returns a healthy 200 response, however as soon as I try a POST request I get: **502 Bad Gateway**

Here is my **app.yaml**:

    service: pdf-service

    

    # [START runtime]

    runtime: nodejs

    vm: true

    # [END runtime]

    

    threadsafe: yes

    

    # Temporary setting to keep gcloud from uploading node_modules

    skip_files:

     - ^node_modules$

    

    handlers:

    - url: (.*)/

      script: app.js

      secure: always

My **app.js** script:

    'use strict';

    

    var express = require('express');

    var app = express();

    var cors = require('cors');

    var bodyParser = require('body-parser')

    var phantom = require('./phantom');

    

    app.use(cors());

    // parse application/x-www-form-urlencoded

    app.use(bodyParser.urlencoded({ extended: false }))

    

    var tmpURL = 'https://www.google.com';

    // [START hello_world]

    // Say hello!

    app.post('/', function(req, res) {

        console.log('requrl', req.body);

        phantom.takeScreenShot(tmpURL, res);

    });

    

    app.get('/', function(req, res) {

      res.status(200).send({'greetings': 'Hello World'});

    });

    // [END hello_world]

    

    if (module === require.main) {

        // [START server]

        // Start the server

        var server = app.listen(process.env.PORT || 8080, function() {

            var host = server.address().address;

            var port = server.address().port;

    

            console.log('App listening at http://%s:%s', host, port);

        });

        // [END server]

    }

    

    module.exports = app;

Notes:

**The code works on my local environment.**

[Nicholas (Google Cloud Support)]

Hello!  Thanks for posting your questions here.  

Your GET and POST handlers are doing very different things in this case so I would not expect to receive the same response from both.  

The 502 Bad Gateway is most likely caused by phantom.takeScreenShot(url, response) and there. I could not find any such method listed in the PhantomJS documentation. I did however find the documentation for the webpage module which provides screen capturing functionalities.

In fact, there appear to be functions to accomplish exactly what you seek if I understood that correctly. The open(url, callback) method issues the HTTP request and invokes the callback when done. The renderBase64(format) method renders the pages content in a given format and outputs the content in base64.

The example provided in their documentation is fairly concise. Here's one with some additional error handling:

// top of your program
var webpage = require('webpage');

// handler for your post request
// issues HTTP request to given URL
// returns base64-rendered screenshot on success
// or error message on failure
function post_handler(request, response) {
  var page = webpage.create();
 
  page.viewportSize = {
    'width': 1920,
    'height': 1080
  };
 
  page.open('https://www.google.com', function (status) {
    var base64;
   
    if (status === 'success') {
      base64 = page.renderBase64('PNG');
      return response.status(200).send({
        'screenshot': base64
      });
    }
   
    response.status(200).send({
      'error': 'Request to ' + page.url + ' returned with status ' + status
    });
  });
}

This seems to be the way their documentation suggests getting screen captures. Let me know if this works for you. If you still encounter errors, I would suggest adding listeners to onResourceError and other error-related events.  If getting unknown errors beyond that, please post some of your request logs from your application as they might have more relevant information.

[Harris Robin Kalash]

Hi,

Thanks for your response.

The phantom file is not the library, it is simply referencing a file with my own method, a method that does what the post_handler you posted is doing. This works perfectly locally. I don't think the issue is with that file. There are other things I could not figure out which leads me to believe my problem might be with my yaml file. For example, after I deploy with :

gcloud -q preview app deploy app.yaml --version 1 --promote

and I have POST handlers such as the following in my code:

app.post('/postTEST', function(req, res) {

    console.log('requrl', req.body);

    res.status(200).send({'greetings': 'You made a POST request'});

});

going to /postTEST returns a 404. However, if this was a GET handler it would work perfectly fine.

You can even test it yourself here:

Do a GET to the following URL :

https://thirdshelf-console-pdf-dot-thirdshelf-console-dev.appspot.com

and then try a POST at :

https://thirdshelf-console-pdf-dot-thirdshelf-console-dev.appspot.com/postTEST

Thanks Nicholas.

[Nicholas (Google Cloud Support)]

The nodejs application should then be working fine based on the information you've provided.

You are however correct that the app.yaml needs to be changed.  The url of your main handler is incorrect to pass requests along to your nodejs application.  Please note that the url property in the YAML is a POSIX extended regular expression.  From the app.yaml you described in your first message, I presume you would like to route all requests to your nodejs application.  To accomplish this, the recommended expression is /.*.  So the secure handlers would look like this:

handlers:
- url: /.*
  script: app.js
  secure: always

This will route all requests to the your nodejs application and redirect to https to respect secure: always.  Let me know if you still encounter this issue.  

On Monday, May 30, 2016 at 9:42:45 PM UTC-4, Harris Robin Kalash wrote:

[Nicholas (Google Cloud Support)]

I have not heard from you in some time.

• Are you still experiencing this issue?
  
• Have you tested your application using the app.yaml suggestion I provided
  
• Are there new errors related to this application?
  

On Monday, May 30, 2016 at 9:42:45 PM UTC-4, Harris Robin Kalash wrote:

[Jeff Payne]

See https://groups.google.com/d/msg/google-appengine/HEH9pjbuvog/ckbccnTkCQAJ for my solution to something similar.