ERROR: (gcloud.compute.ssh) Could not fetch resource: - Insufficient Permission: Request had insufficient authentication scopes.

5,081 views
Skip to first unread message

John Fields

unread,
Sep 17, 2019, 11:24:51 PM9/17/19
to google-dl-platform
I've been working with my software engineer for a week to diagnose the SSH issues when trying to use gcloud compute commands with our new Google Cloud Deep Learning VM image.  Since we can't find any support through Google documentation, Slack, Stack Overflow, etc I'm meeting with our CTO tomorrow to suggest that we should consider another solution to Google Cloud since there is no support available.

Viacheslav Kovalevskyi

unread,
Sep 18, 2019, 11:47:30 AM9/18/19
to John Fields, Yixin Shi, google-dl-platform
thank you John for reaching out to us. Adding +Yixin Shi who should be able to drive this, meanwhile may I ask you to:

* provide, is possible serial port logs from the VM, this will allow us to start the investigation

Our DLVMs enabling SSH server after all the initial scripts are executed, therefore time to time you just need to wait a little bit, or maybe one of our startup scripts actually failed to start (serial logs should help)  

On Tue, Sep 17, 2019 at 8:24 PM John Fields <john....@harqen.com> wrote:
I've been working with my software engineer for a week to diagnose the SSH issues when trying to use gcloud compute commands with our new Google Cloud Deep Learning VM image.  Since we can't find any support through Google documentation, Slack, Stack Overflow, etc I'm meeting with our CTO tomorrow to suggest that we should consider another solution to Google Cloud since there is no support available.

--
You received this message because you are subscribed to the Google Groups "google-dl-platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-dl-platf...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-dl-platform/c0c59327-e1ee-4c2e-82b6-036e2795b8ae%40googlegroups.com.


--
WBR,

Viacheslav Kovalevskyi

Key priorities for Data Science Experience Team in Q3: 
* make Notebooks FinServ compliant
* Horizontals for Notebooks, DLVMs, DL Containers 
* Notebooks Scheduler
Message has been deleted

John Fields

unread,
Sep 20, 2019, 11:08:35 PM9/20/19
to google-dl-platform
Thank you for the help to resolve the SSH issues.  Unfortunately, we are now experiencing issues with Jupyter Notebook (trying to connect to Google Colab).  We followed the instructions but they do not work...

Connecting to Jupyter Lab

A Jupyter Lab session is started when your Deep Learning VM instance is initialized. To access it, set up port forwarding on your instance:

export INSTANCE_NAME="my-instance"
gcloud compute ssh $INSTANCE_NAME -- -L 8080:localhost:8080
This is the error message when we execute the above command:
bind: Address already in usechannel_setup_fwd_listener_tcpip: cannot listen to port: 8080Could not request local forwarding.

When we check for 8080 use, it's Jupyter notebook listening but if we try http://localhost:8080 we receive a "localhost refused to connect" error message.
$ sudo lsof -i tcp:8080

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
jupyter-l 3169 jupyter 4u IPv6 132583 0t0 TCP *:http-alt (LISTEN)
jupyter-l 3169 jupyter 5u IPv4 132584 0t0 TCP *:http-alt (LISTEN)

PLEASE HELP!
Message has been deleted

Mike Cheng

unread,
Sep 23, 2019, 5:31:49 PM9/23/19
to John Fields, google-dl-platform
Hey John,

You shouldn't ever need to create a jupyter server locally on your Mac. Your SSH port forwarding issue (bind: Address already in usechannel_setup_fwd_listener_tcpip: cannot listen to port: 8080Could not request local forwarding.) is caused by port 8080 already being in use on your local machine, so you should kill the local jupyter instance using that port. Then, run the port forwarding and connect Colab to local port.

Best,
Mike

On Sat, Sep 21, 2019 at 9:54 AM John Fields <john....@harqen.com> wrote:
I spent about 6 hours to finally get Google Colab/Jupyter notebook to work with our Deep Learning VM.   There are at least two sets of instructions and neither had the right instructions for me.  Here are the steps I used to finally get it to work:

1. From the terminal on my Mac:
jupyter notebook \
  --NotebookApp.allow_origin='https://colab.research.google.com' \
  --port=8888 \
  --NotebookApp.port_retries=0

2. From the terminal on my Mac (this doesn't work if you click the SSH button in the Google Console SSH :
gcloud compute ssh "instance name" --zone "zone" -- -L 8888:localhost:8888 

3. Start Colab and connect to local port 8888

Very, very frustrated with how hard everything is with Google Cloud DL image.

--
You received this message because you are subscribed to the Google Groups "google-dl-platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-dl-platf...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages