SSH and WinSCP into VM instance: Can only browse, but can not view, download or upload files

[Ishwara Bhat]

Hi,

I got the GCE VM (n1-standard-1) instance running, configured SSH and able to run Win SCP and brose the VM instance folders. But I can not open any file from the VM. Can not download or upload.

What should I do to have the ability to upload/download? Please suggest what could be wrong or what settings are to be changed for this. 

Thanks

Ishwar

[Dinesh (Google Platform Support)]

Hello Ishwara,

There are several ways to upload/download/transfer files to VM instances on GCE. It depends on whether you are using Linux instances or Windows instances.  

1. For Linux instances, use the gcloud command-line tool. The gcloud compute scp command obtains the necessary private SSH keys for you from the project and instance metadata and then uses the SCP tool to perform the file transfer.

2. For Window instances, you can use RDP client.

3. You can use Google Cloud Storage to transfer files between instances or your local computer. If you're creating many instances and uploading the same set of files to them, it would be easier to upload your files to a GCS bucket and then use 'gsutil cp' command in your script to automate the transferring the files to your VMs
   

4. Moreover, you can use the file browser on Linux or macOS workstations to transfer files. Similarly, you can use WinSCP for windows workstation.

You can use CAT command on SSH terminal to view a file on your VM.

There could be many reasons that you are not able to view or transfer the file and few of them are:

i.) You don’t have permission to access the file or drive

ii.) Wrong SSH keys

iii.) Wrong destination or source name or IP address in file transfer command.  

[Ishwara Bhat]

I have SSH and keys setup. Hence i am able to login and browse the instance folders. The issue is after that i am not able to change the permissions of specific folders. Chmod 777 gets "permission denied".. Hence I thought there is some other superuser account to which I should elevate...su <google user name that created instance>. It needs a password. What password to use? In GCE metadata, we have the full key listed. Should we use the key as password in SSH command line password prompt?

[Dinesh (Google Platform Support)]

Have you been using the sudo command?

You can either change the permission of the directory or change the ownership of directory with the sudo command in linux.

You can refer to this stack exchange discussion for further clarifications on permission denied issues.   

[Ishwara Bhat]

Now I tried sudo (On the SSH on VM instance and also SSH on project level) .
$sudo su USR NAME it appears to work. Does not return error.
But when I do chmod 777 after that, it says "operation not permitted". This is the case with both SSH on VM instance and SSH on project level shell.

I am trying this on SSH after logging into cloud, not from gcloud command line.

However, I noticed that the shell that comes up shows that the login name that shows in SSH is different that the owner gmail account.

Also I had created the SSH key on windows desktop (with WinSCP). Then added the key to SSH private key under VM instance SSH key.

i.e. I did not create key with the below command.

sudo su USERNAME
ssh-keygen -t rsa -f ~/.ssh/my-ssh-key -C USERNAME

Is this a problem?

[Dinesh (Google Platform Support)]

I think we lost the track here. Can you please clarify your objective here?

Initially, you asked about how to open and upload/download files from VM and I recommended you different ways of file transfer. Can you please confirm which option you are following and at which step you are getting an error message?

If a user requires root permissions, they can get those permissions by running commands using sudo. Please note that user has to use prefix “sudo” with every command to get elevated permissions.

If you are trying to use WinSCP from windows client to access VM terminal, you need to add SSH key to the metadata. You can follow how to create a new SSH key for the windows client.

[Ishwara Bhat]

I am trying to magento on VM on GCE. Magento is setup. Now I need to add theme. For this, i need to upload a set of files. I have the files in my local disk. I need to upload using WinSCP. For this connection to happen, I generated SSH key from WinSCP puttyGen tool on laptop and added that key under metadata of VM instance. (I did not do this from gcloud ).

Using this key, I am able to login to VM instance and browse the directories in the VM. But when I upload the theme folder+files, i get the error. ( All the while I am talking about credentials of SSH, not gcloud SDK). 

Thanks

Ishwar

[Dinesh (Google Platform Support)]

I believe you are transferring the files directly to magento folder on the VM instances and you don’t have root permission for that folder. One workaround would be to transfer files to a folder where you have permissions (for example /home/username or /tmp etc), then log in via ssh and use sudo to move/copy files to their final destination. Further, you can refer this thread.

[Ishwara Bhat]

Thanks a lot. I will try it.

In that thread I also saw "Another solution would be to change permissions/ownership of the directories you uploading the files to, so your non-privileged user is able to write to those directories."

how do i do it? I have instance-level SSH key. For this, is there a way to setup ownership permissions? 

Meanwhile, I am very confused with IAM & admin roles, then the metadata within VM instances. In IAM & admin, i see option to set the roles. Within VM instance, i see 'metadata' to add SH keys. But there is no other roles and permission setup within VM instances. Also I suspect that I created many SSH keys. When I login to shell, i am not able to understand which user name is logged in. I think  I may have messed up the user accounts and keys. Is it possible to review my project and sugegst me if i have done something wrong? Thanks a lot.

[Dinesh (Google Platform Support)]

As per one of my previous post (stack exchange discussion), you can refer the following command to change permission of the directories.

sudo chmod -R 777 /home/sixven/camp_sms/inputs

The following command to change ownership (and make user:francisco-vergara as the owner) of the directories. I would suggest changing the ownership of the directory since it is certainly more secure.

sudo chown -R francisco-vergara:francisco-vergara /home/sixven/camp_sms/inputs

I strongly suggest going through these steps for transferring files using WinSCP

To understand how GCE manages the ssh keys and how GCE manages the metadata, please go through this article about Google Cloud SSH Keys.Google Cloud Identity and Access Management allow users to create and manage permissions for GCP resources which include users project resources, GCE, GAE, API, cloud tasks, cloud trace etc.  That being said,  SSH keys created in your project, will let you access the VM, but you cannot change the permissions of the directory of the VM via the IAM console.

The purpose of instances-level SSH keys is to authentication users and initiate command line session with Linux instances. They used for specific instances only and it has no significance with respect to VM directories permissions or ownership. You can refer this link to manage ssh keys

When you SSH in a VM, Linux kernels are configured by default to show the user you have logged in.  i.e.

username@instance_name

You can also verify the user you have logged in via traditional linux commands. I.e

username@instance_name:~$ who i am

username     pts/0        2017-10-17 17:56

In that sense, I really do not believe there could be something wrong with the definition of the keys in your project. Instead I think your problem is related to the permissions in the VM.  Nevertheless, if you feel there is something wrong in your project metadata, feel free to send me via a private message your project ID.

[Ishwara Bhat]

Thanks.

Overall closure:

1. After creating instance in the UI of cloud.google.com, I created SSH. For this, I started at my laptop WinSCP application. I ran PuttyGen to create SSH keys (Did not use the online Google cloud option, but the winSCP tool). It got me the key pair. I used the private key file locally. The key pasted in SSH key under meta data of the instance. 

So, my SSH is now setup. I could SSH to VM instance.

Google cloud allows read/write access only to the SSH key user's /home folder. 

I uploaded the files to that home folder.

Later used GCLOUD command line tools from the laptop to "$gcloud compute ssh" into the VM instance. Now I am the super user. I can do stuffs which are not allowed in normal SSH user. I could copy files from /home folder of the previous SSH user home folder to the folder I want. Then I could run the commands I want.

Summary: As a normal SSH user of winSCP, not being to upload to required folder is a normal behavior. Need gcloud tools to do critical folder access. There is no alternative to gcloud command line.

Hope this is correct summary.