What script does "Deploy" button in Deploy Heroku heading running?

[Lena Yemelyanov]

I'm trying to troubleshoot the issue #316, essentially something goes wrong with the deploy script when you fill out the form to deploy the app.  The exact error message is described in the incident.  I've forked the repo and I want to see if my fix works but I'm not entirely sure how to get an instance of the deploy form that calls on my bug fix branch.  Can anyone help?

[Andrew Fischer]

Hi there,

I believe the Heroku deploy uses the command described in the Procfile on boot: ./bin/install_customizations && npm run build && npm start

It's very possible we have a package-lock out of sync issue. In theory, Library supports Node >= v10, but it's possible a package-lock change was committed when developing with a version of Node > v10.

Would you be able to provide the full error Heroku is logging when trying to deploy in your issue? I can try to reproduce later, but I think I also need to recover my Heroku account due to the recent security breach...

Thanks,

Andrew

[Lena Yemelyanov]

Hey Andrew, 

Here's everything I know: https://github.com/nytimes/library/issues/316

I tried deploying it using Google App Engine instead, because I think the issue is that Heroku is blocking Gitlab deployment right now (see the issue above) and I'm now struggling with the Uh Oh 500.  what I did have to do is change the following in the package.json from 

  "engines": {
    "node": ">=10.x",
    "npm": ">=6.5.x"
  }

to 

  "engines": {
    "node": "10.x",
    "npm": ">=6.5.x"
  }

[Lena Yemelyanov]

Hi Andrew,

One more question.  This might be a very basic one, I'm totally new to any sort of web development.  I couldn't find instructions on how to set the session secret in the .env for the Google Cloud deployment.  So I set it to this: SESSION_SECRET=secretvalue  per earlier instructions in development, if I had to guess that is the reason I'm getting "uh, oh that's a 500". Do you have some advice on how to set that variable?

[Andrew Fischer]

Your session secret should be set to any reasonable random value. I would recommend using a password manager to generate one. This value is used to encrypt cookies to make sure a user's connection to a server is secure. For a longer explanation (with some pretty technical deep dives, if you're really interested), check out this article.

I unfortunately don't think that the 500 would be caused from this value though. Most commonly, 500s in this app are caused by Google Drive permission issues. Are you able to view and share any logs from google cloud? If you're using app engine, you should be able to view recent logs from the "Tools" menu in the services tab, or follow these instructions. Any errors you see there would be super helpful!

-Andrew

[Lena Yemelyanov]

thank you for offering to look at the logs! this escalated quickly. lol. 

After looking at everything further, the error is probably here:

2022-05-17 22:28:12.375 EDT

info: Trying to parse client credentials via GOOGLE_APPLICATION_JSON.

2022-05-17 22:28:12.375 EDT

warn: failed updating tree

2022-05-17 22:28:12.375 EDT

{ message: 'Unexpected token \' in JSON at position 0',

so something to do with my private key for the service account...Logs

[Lena Yemelyanov]

Also it's just a demo so, here's the exact .env I'm using, I plan to make a different service account for the actual real life app, etc.

APPROVED_DOMAINS=gmail.com
DRIVE_ID=1j08XNXcW8k7xS9a9foxDGf9OTyM36Zoc
DRIVE_TYPE=folder
GOOGLE_APPLICATION_JSON='{
  "type": "service_account",
  "project_id": "pk-move-wiki-demo",
  "private_key_id": "63e3e977f4dfceb7194efa6cf268db7e43094f78",
  "private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQDeB/JuzySkaebU\nNoRrDK85R1Gw2PlmeQXWiy9HVylGi7JYbxwsfE+nyWlDkE/xmybhjEsyYTX1hee2\nRTjNMjzQMHE4bYN2ZzMyp4BYHl1HZIXDekXZO4JomFH3q4kcW4ZQOUFfoqiuuBUw\n4jTmRP2IkCDD9Ty1/d13VCxre6tYgKSVNiABOFigOSK98ra5cn//RQaZkvu2TMDX\nmPbrlsXaw/beUY8d7iZV4HLAhNTiP8mPRaW6mYLC5/9tnwiOtNFT8XyTUvxlLOHD\nx0hRmJMOW2mnLLGB9AXHmNWio6Jt7jUpnoucsd/wL4ODdHrVaF6eKF0If21Av/Cg\nYTb/ghpNAgMBAAECggEAMlqdCbJivgLifbhLeberuZLKHRBoJ2xE/a4dL9G9o6kk\n7ldsjU44BnZwMAl3h/TG8ZoOp8NSKHeQ4E58nlTJY/HIi6KSkPTAsLk6K0Hfp8RN\nbdK6KpULc0swM7osY0kCggkqIVVTaiSHccmrc5mr4h/FvotMlH2Zj2mAlXO7nVtL\nTfQwhMQcVofATHLXk0KYpiT9EuTkGEwqUq2AbXjVbcNsekURhOCqPkcsnYF4jAPA\nPAYeF6mpkdaQt8yJdFb9AvkTmL7wXWeMFDLWS1KvRawm/RQl9GjfzLzSxSAGBw1d\nxmkqH38qjx5GsFK2oHo0mBc+Bc89Lw+WCC7MRjS1xQKBgQDzp6wvXeSWV77a7WI/\ndFhrbBUUu6plxBsloAuhjW7j3SfXYfLNGYjeZFM6yjIjJJj9V+t8mbXXxytSko98\n4yWD8W08X4Sepki5ipOtE+7uDxGAuYD+nsoOHjs/wctSzdQ9kDrZDc/yVJDL+EJu\nYMzujn26KQSD0AS0MomtSTTC+wKBgQDpR80a+a1oboqGk1A8vbfsJ5zQVXGNDiTQ\nwHfwgS9TQzKAW9eE0kDiemOSpubao7CccNb7F7IphPDfG+eU4xl3jL+Pc/TKffDt\n0kFcISPfcwJicIlAEA11SGJC+uz1v9akQZUn0ydFW/IeaYhtgmOl5fPwhIzcAB/+\nppA9cbvVVwKBgAWyGbhRvZI2IuQDQJSzJ437TNLbQDR4zRmoEB2KrNjuxoumYoKg\n4i4wsjFhoKJprJnfgEW9Yh7VLRIVEv2cYbgf4KnUockDoheUQbd+C8vM1l/n9jf9\nhkOmOURYR2R8I0X9JSMFy6Kij2/1Mbha0u+QrSe45b4fEr0ToVxHpdAZAoGAF5FO\ncmQ5FazdR91IJOrR0wTpfc60CxkbtQ9Mzrpuao4GYxtkOaH4NilBOAaumfYhmJx5\ny6JIOJOath+elNcgsohAriB74Z05ov0z5zfd5Ow8mG9gHl3AzlGCw9uP5v/7klqh\n1AUsGYDBhDsLMm0S3ibOL0Vk6bEFinePaPW9kT8CgYB7vI1NfIYBhVhauDCL1E+Z\nNM+TFZVd4Fi/kX2S1W6ewbDdBuGzLzZgFfEsz5IVI4Nv3H2AEse50hlgcXoKR305\nxFby8/QS8pE+l8UCk7PcYvRomaih8K7TzlX1KU5vrHJlymn+b9exmGMQd64Yj+9h\n8tod9tIlTE4SalVt1XkGaw==\n-----END PRIVATE KEY-----\n",
  "client_email": "pkmovew...@pk-move-wiki-demo.iam.gserviceaccount.com",
  "client_id": "116207943625012389241",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/pkmovewikidemo%40pk-move-wiki-demo.iam.gserviceaccount.com"
}
'
GCP_PROJECT_ID=pk-move-wiki-demo
GCP_PROJECT_NUMBER='714896530502'
GOOGLE_APPLICATION_CREDENTIALS=parse_json
GOOGLE_CLIENT_ID=714896530502-n9d4marrhv3mavobe5nersjekn9et7k3.apps.googleusercontent.com
GOOGLE_CLIENT_SECRET=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
SESSION_SECRET=secretvalue

[Lena Yemelyanov]

Ok.  I got this all sorted out.  I think the issue is that  GOOGLE_APPLICATION_JSON has to be o a single line and it CANNOT contain quotes.  It would help to update the instructions here: https://nyt-library-demo.herokuapp.com/get-started/deploying-library-google-app-engine to exclude the code and specify that when the service account creates keys json that file must be modified to be a single line.

[Lena Yemelyanov]

*exclude the quotes (not code)

[Carl M. Johnson]

I have in the past had problems with the line breaks and quotes in GOOGLE_APPLICATION_JSON getting mangled for my other apps, so a workaround I've used is to base64 encoded it first and then decode it in my app before passing it on to Google. I don't know, but maybe that would be nice feature to add to Library.

— Carl Johnson 

[Andrew Fischer]

Thanks, I've added a line to the docs about making sure that JSON is on a single line. Appreciate your troubleshooting.

Be sure to deactivate those google credentials you posted, @lenay21!

-Andrew

[Uri Blackman]

I'm getting the CLI error deploying to Heroku, even though the security incident referred to seems to be fixed based on https://github.com/nytimes/library/issues/316

My error log is pasted below.

Any ideas?

-----> Building on the Heroku-22 stack
-----> Determining which buildpack to use for this app
-----> Node.js app detected
       
-----> Creating runtime environment
       
       NPM_CONFIG_LOGLEVEL=error
       NODE_VERBOSE=false
       NODE_ENV=production
       NODE_MODULES_CACHE=true
       
-----> Installing binaries
       engines.node (package.json):  >=10.x
       engines.npm (package.json):   >=6.5.x
       
       Resolving node version >=10.x...
       Downloading and installing node 18.10.0...
       Bootstrapping npm >=6.5.x (replacing 8.19.2)...
       npm 8.19.2 installed
       
-----> Installing dependencies
       Installing node modules
       npm ERR! code EUSAGE
       npm ERR!
       npm ERR! `npm ci` can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with `npm install` before continuing.
       npm ERR!
       npm ERR! Missing: m...@2.1.3 from lock file
       npm ERR!
       npm ERR! Clean install a project
       npm ERR!
       npm ERR! Usage:
       npm ERR! npm ci
       npm ERR!
       npm ERR! Options:
       npm ERR! [-S|--save|--no-save|--save-prod|--save-dev|--save-optional|--save-peer|--save-bundle]
       npm ERR! [-E|--save-exact] [-g|--global] [--global-style] [--legacy-bundling]
       npm ERR! [--omit <dev|optional|peer> [--omit <dev|optional|peer> ...]]
       npm ERR! [--strict-peer-deps] [--no-package-lock] [--foreground-scripts]
       npm ERR! [--ignore-scripts] [--no-audit] [--no-bin-links] [--no-fund] [--dry-run]
       npm ERR! [-w|--workspace <workspace-name> [-w|--workspace <workspace-name> ...]]
       npm ERR! [-ws|--workspaces] [--include-workspace-root] [--install-links]
       npm ERR!
       npm ERR! aliases: clean-install, ic, install-clean, isntall-clean
       npm ERR!
       npm ERR! Run "npm help ci" for more info
       
       npm ERR! A complete log of this run can be found in:
       npm ERR!     /tmp/npmcache.jPIPu/_logs/2022-09-29T02_43_27_953Z-debug-0.log
-----> Build failed
       
       We're sorry this build is failing! You can troubleshoot common issues here:
       https://devcenter.heroku.com/articles/troubleshooting-node-deploys
       
       Some possible problems:
       
       - Dangerous semver range (>) in engines.node
         https://devcenter.heroku.com/articles/nodejs-support#specifying-a-node-js-version
       
       Love,
       Heroku
       
 !     Push rejected, failed to compile Node.js app.
 !     Push failed

[Uri Blackman]

I solved this problem by deploying on Google Cloud and updating the files for the node.js version from 10 to 16.

specifically:

package.json:

"engines": {
"node": ">=16.0",
"npm": ">=6.5.x"
},

and in app.yaml

runtime: nodejs16
instance_class: F4